Re: [PATCH 3/5] build: optionally build python-cffi bindings
Tomi Ollila writes: > how bad does out-of-tree build break with this -- do we need to do > the same as with ruby bindings (copy sources -- do we still do so)? or does > python provide better alternative..? Good question. What is here will break with out of tree builds, just from trivial things like missing $(srcdir). At a deeper level someone would probably have to modify the build bits (python-cffi/setup.py, python-cffi/notb/_build) to work out of tree. I don't know how well any of this is supported by the setuptools module. It seems more friendly to the idea of building inside the source tree with destination outside, but I'm really not a setuptools expert. d ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH 4/5] tests: run python-cffi tests
On Mon, Nov 04 2019, Tomi Ollila wrote: > > as for checking pytest existence, the following works somewhat: > >~/test-venv/bin/python3 -m pytest --version > > It prints the version, then exists. Could not figure out how to request > specific version in quick look (10 min search/trial/error session), > parsing version output is so... :/ Hah, dumped ~/test-venv/. so retry... $ python3 -m venv ~/test-venv $ ~/test-venv/bin/pip3 install pytest $ ~/test-venv/bin/python3 >>> import pytest >>> dir(pytest) >>> pytest.__version__ prints out '5.2.2'. argh, string again (in addition to resorting to __...__ names/attributes of whatever these are called in python ;). anyway, even parsing this version string in python than doing it in shell script would be better...(*) anything better ? Tomi (*) got exhausted when looking through this: https://stackoverflow.com/questions/11887762/how-do-i-compare-version-numbers-in-python ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH 4/5] tests: run python-cffi tests
On Sun, Nov 03 2019, David Bremner wrote:
> The entire python-cffi test suite is considered as a single test at
> the level of the notmuch test suite. This might or might not be ideal,
> but it gets them run.
> ---
> test/T391-python-cffi.sh | 12
> 1 file changed, 12 insertions(+)
> create mode 100755 test/T391-python-cffi.sh
>
> diff --git a/test/T391-python-cffi.sh b/test/T391-python-cffi.sh
> new file mode 100755
> index ..a03f9e0e
> --- /dev/null
> +++ b/test/T391-python-cffi.sh
> @@ -0,0 +1,12 @@
> +#!/usr/bin/env bash
> +test_description="python bindings (pytest)"
> +. $(dirname "$0")/test-lib.sh || exit 1
> +
> +if [ $NOTMUCH_HAVE_PYTHON_CFFI -eq 0 ]; then
> +test_done
> +fi
> +
> +
> +test_begin_subtest "python cffi tests"
> +test_expect_success "(cd $NOTMUCH_SRCDIR/bindings/python-cffi/build/stage &&
> ${NOTMUCH_PYTHON} -m pytest --log-file=$TMP_DIRECTORY/test.output)"
probably
test_expect_success "${NOTMUCH_PYTHON} -m pytest \
--log-file=$TMP_DIRECTORY/test.output \
$NOTMUCH_SRCDIR/bindings/python-cffi/build/stage"
worked (which reminds me out-of-tree... ;)
as for checking pytest existence, the following works somewhat:
~/test-venv/bin/python3 -m pytest --version
It prints the version, then exists. Could not figure out how to request
specific version in quick look (10 min search/trial/error session),
parsing version output is so... :/
> +test_done
> --
> 2.24.0.rc1
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH 3/5] build: optionally build python-cffi bindings
On Sun, Nov 03 2019, David Bremner wrote:
> Put the build product (and tests) in a well known location so that we
> can find them e.g. from the tests.
> ---
> Makefile.local | 2 +-
> bindings/Makefile.local | 9 +
> 2 files changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/Makefile.local b/Makefile.local
> index 3c6dacbc..7c12612d 100644
> --- a/Makefile.local
> +++ b/Makefile.local
> @@ -1,7 +1,7 @@
> # -*- makefile -*-
>
> .PHONY: all
> -all: notmuch notmuch-shared build-man build-info ruby-bindings
> +all: notmuch notmuch-shared build-man build-info ruby-bindings
> python-cffi-bindings
> ifeq ($(MAKECMDGOALS),)
> ifeq ($(shell cat .first-build-message 2>/dev/null),)
> @NOTMUCH_FIRST_BUILD=1 $(MAKE) --no-print-directory all
> diff --git a/bindings/Makefile.local b/bindings/Makefile.local
> index 18f95835..b8e18c92 100644
> --- a/bindings/Makefile.local
> +++ b/bindings/Makefile.local
> @@ -13,6 +13,13 @@ ifeq ($(HAVE_RUBY_DEV),1)
> $(MAKE) -C $(dir)/ruby
> endif
>
> +python-cffi-bindings: lib/$(LINKER_NAME)
> +ifeq ($(HAVE_PYTHON_CFFI),1)
> + cd $(dir)/python-cffi && \
how bad does out-of-tree build break with this -- do we need to do
the same as with ruby bindings (copy sources -- do we still do so)? or does
python provide better alternative..?
> + ${PYTHON} setup.py build --build-lib build/stage && \
> + mkdir -p build/stage/tests && cp tests/*.py build/stage/tests
> +endif
> +
> CLEAN += $(patsubst %,$(dir)/ruby/%, \
> .RUBYARCHDIR.time \
> Makefile database.o directory.o filenames.o\
> @@ -20,3 +27,5 @@ CLEAN += $(patsubst %,$(dir)/ruby/%, \
> status.o tags.o thread.o threads.o)
>
> CLEAN += bindings/ruby/.vendorarchdir.time
> +
> +CLEAN += bindings/python-cffi/build
> --
> 2.24.0.rc1
>
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH 2/5] configure: check for python cffi module
On Sun, Nov 03 2019, David Bremner wrote:
> This is needed to build the new python bindings
> ---
> configure | 15 +++
> 1 file changed, 15 insertions(+)
>
> diff --git a/configure b/configure
> index 3c148e12..300591fe 100755
> --- a/configure
> +++ b/configure
> @@ -671,6 +671,15 @@ if [ $have_python -eq 0 ]; then
> errors=$((errors + 1))
> fi
>
> +printf "Checking for python cffi... "
> +if "$python" -c 'import cffi' >/dev/null 2>&1; then
> +printf "Yes.\n"
> +have_python_cffi=1;
trailing ;
> +else
> +printf "No.\n"
> +have_python_cffi=0;
ditto :)
otherwise tolerable =D
> +fi
> +
> printf "Checking for valgrind development files... "
> if pkg-config --exists valgrind; then
> printf "Yes.\n"
> @@ -1223,6 +1232,9 @@ HAVE_GETLINE = ${have_getline}
> # building/testing ruby bindings.
> HAVE_RUBY_DEV = ${have_ruby_dev}
>
> +# Is the python cffi package available?
> +HAVE_PYTHON_CFFI = ${have_python_cffi}
> +
> # Whether the strcasestr function is available (if not, then notmuch will
> # build its own version)
> HAVE_STRCASESTR = ${have_strcasestr}
> @@ -1376,6 +1388,9 @@ NOTMUCH_RUBY=${RUBY}
> # building/testing ruby bindings.
> NOTMUCH_HAVE_RUBY_DEV=${have_ruby_dev}
>
> +# Is the python cffi package available?
> +NOTMUCH_HAVE_PYTHON_CFFI=${have_python_cffi}
> +
> # Platform we are run on
> PLATFORM=${platform}
> EOF
> --
> 2.24.0.rc1
>
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH 4/5] tests: run python-cffi tests
Tomi Ollila writes: > On Sun, Nov 03 2019, David Bremner wrote: > >> The entire python-cffi test suite is considered as a single test at >> the level of the notmuch test suite. This might or might not be ideal, >> but it gets them run. > > IMO this is good enough approach -- provided that pytest is prereq > for this test to be run =D > > (i.e. one may have cffi but decided not to install pytest) > > Tomi Yes, that's a good point, that actually caused some failures. I guess I'll also need to check for pytest-cov, or disable coverage testing, which is apparently enabled by default in tox.ini. At the moment I don't know the best way to do that, but "python3 -m pytest -c /dev/null" seems to work. d ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH 4/5] tests: run python-cffi tests
On Sun, Nov 03 2019, David Bremner wrote:
> The entire python-cffi test suite is considered as a single test at
> the level of the notmuch test suite. This might or might not be ideal,
> but it gets them run.
IMO this is good enough approach -- provided that pytest is prereq
for this test to be run =D
(i.e. one may have cffi but decided not to install pytest)
Tomi
> ---
> test/T391-python-cffi.sh | 12
> 1 file changed, 12 insertions(+)
> create mode 100755 test/T391-python-cffi.sh
>
> diff --git a/test/T391-python-cffi.sh b/test/T391-python-cffi.sh
> new file mode 100755
> index ..a03f9e0e
> --- /dev/null
> +++ b/test/T391-python-cffi.sh
> @@ -0,0 +1,12 @@
> +#!/usr/bin/env bash
> +test_description="python bindings (pytest)"
> +. $(dirname "$0")/test-lib.sh || exit 1
> +
> +if [ $NOTMUCH_HAVE_PYTHON_CFFI -eq 0 ]; then
> +test_done
> +fi
> +
> +
> +test_begin_subtest "python cffi tests"
> +test_expect_success "(cd $NOTMUCH_SRCDIR/bindings/python-cffi/build/stage &&
> ${NOTMUCH_PYTHON} -m pytest --log-file=$TMP_DIRECTORY/test.output)"
> +test_done
> --
> 2.24.0.rc1
>
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch
Re: python CFFI bindings integration into notmuch build/test
David Bremner writes: > This series is on top of a slightly modified version [1] of > > id:20191008210312.20685-2-f...@devork.be > > If you want to try out the new bindings, you can try the branch wip/cffi in > the repo > https://git.notmuchmail.org/git/notmuch > > One current annoyance is that the version of pytest on travis is too > old for what is specified in tox.ini. > I fixed this by moving the travis env to bionic (2018.x). This involved dropping dkg's notmuch ppa. Daniel, do you happen to know if there is anything in that repo that we should still keep when building in ubuntu bionic? FWIW, the build succeeds without that PPA. d ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: notmuch-mode and org-capture
Tristan Cacqueray writes: >> Hmm I'm on 9.2.3 for the time being, and there appears to be no change >> to anything named `notmuch` between 9.2.3 and 9.2.6. >> >> Is the change you're talking of unreleased yet? > > Indeed the change isn't actually released, the rename happened in: > > https://code.orgmode.org/bzg/org-mode/commit/499c0a50cc4b11e37b91374af23cb27ab8fc20d2 > > Thus requiring org-notmuch should work for 9.2.3. Having checked a bit more in-depth the code, thanks to your pointers, it looks like the addition of notmuch-tree to the org-notmuch-store-link function hasn't made it into released code. I've just tried using the stupidest possible patch by just adding notmuch-tree-mode, and it looks like it's not enough. Guess I'll wait for 9.3, then, thank you for your help! ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: Python3 cffi bindings
Hi,
I just checked out the wip/cffi branch on git.notmuch.org with the
purpose of porting Lieer (https://github.com/gauteh/lieer). There
seems to be some missing functionality: `Database.get_directory()`
specifically. I also ran into a couple of warning when building
(included below).
Thanks for your work.
By the way, it does not seem that the API is very far from the
previous python API. If it is close enough, perhaps it is possible to
get away with a bug version bump in the bindings rather than creating
a new package. I understand the need for a new package, but it would
be nice if we could avoid the future confusion of two python binding
packages (if at all possible).
Regards, Gaute
~/dev/notm/notmuch/bindings/python-cffi wip/cffi
notm ❯ python setup.py build
warning: no previously-included files found matching 'setup.pyc'
warning: no previously-included files matching 'yacctab.*' found under
directory 'tests'
warning: no previously-included files matching 'lextab.*' found under
directory 'tests'
warning: no previously-included files matching 'yacctab.*' found under
directory 'examples'
warning: no previously-included files matching 'lextab.*' found under
directory 'examples'
zip_safe flag not set; analyzing archive contents...
pycparser.ply.__pycache__.lex.cpython-37: module references __file__
pycparser.ply.__pycache__.lex.cpython-37: module MAY be using
inspect.getsourcefile
pycparser.ply.__pycache__.yacc.cpython-37: module references __file__
pycparser.ply.__pycache__.yacc.cpython-37: module MAY be using
inspect.getsourcefile
pycparser.ply.__pycache__.yacc.cpython-37: module MAY be using inspect.stack
pycparser.ply.__pycache__.ygen.cpython-37: module references __file__
Installed
/home/gauteh/dev/notm/notmuch/bindings/python-cffi/.eggs/pycparser-2.19-py3.7.egg
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/notdb
copying notdb/_message.py -> build/lib.linux-x86_64-3.7/notdb
copying notdb/__init__.py -> build/lib.linux-x86_64-3.7/notdb
copying notdb/_database.py -> build/lib.linux-x86_64-3.7/notdb
copying notdb/_errors.py -> build/lib.linux-x86_64-3.7/notdb
copying notdb/_tags.py -> build/lib.linux-x86_64-3.7/notdb
copying notdb/_thread.py -> build/lib.linux-x86_64-3.7/notdb
copying notdb/_query.py -> build/lib.linux-x86_64-3.7/notdb
copying notdb/_build.py -> build/lib.linux-x86_64-3.7/notdb
copying notdb/_base.py -> build/lib.linux-x86_64-3.7/notdb
running build_ext
generating cffi module 'build/temp.linux-x86_64-3.7/notdb._capi.c'
creating build/temp.linux-x86_64-3.7
building 'notdb._capi' extension
creating build/temp.linux-x86_64-3.7/build
creating build/temp.linux-x86_64-3.7/build/temp.linux-x86_64-3.7
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3
-Wall -fPIC -I../../lib
-I/home/gauteh/.pyenv/versions/3.7.4/envs/notm/include
-I/home/gauteh/.pyenv/versions/3.7.4/include/python3.7m -c
build/temp.linux-x86_64-3.7/notdb._capi.c -o
build/temp.linux-x86_64-3.7/build/temp.linux-x86_64-3.7/notdb._capi.o
build/temp.linux-x86_64-3.7/notdb._capi.c: In function
‘_cffi_d_notmuch_database_add_message’:
build/temp.linux-x86_64-3.7/notdb._capi.c:980:3: warning:
‘notmuch_database_add_message’ is deprecated: function deprecated as
of libnotmuch 5.1 [-Wdeprecated-declarations]
return notmuch_database_add_message(x0, x1, x2);
^~
In file included from build/temp.linux-x86_64-3.7/notdb._capi.c:495:0:
../../lib/notmuch.h:637:1: note: declared here
notmuch_database_add_message (notmuch_database_t *database,
^~~~
build/temp.linux-x86_64-3.7/notdb._capi.c: In function
‘_cffi_f_notmuch_database_add_message’:
build/temp.linux-x86_64-3.7/notdb._capi.c:1033:3: warning:
‘notmuch_database_add_message’ is deprecated: function deprecated as
of libnotmuch 5.1 [-Wdeprecated-declarations]
{ result = notmuch_database_add_message(x0, x1, x2); }
^
In file included from build/temp.linux-x86_64-3.7/notdb._capi.c:495:0:
../../lib/notmuch.h:637:1: note: declared here
notmuch_database_add_message (notmuch_database_t *database,
^~~~
gcc -pthread -shared -L/home/gauteh/.pyenv/versions/3.7.4/lib
-L/home/gauteh/.pyenv/versions/3.7.4/lib
build/temp.linux-x86_64-3.7/build/temp.linux-x86_64-3.7/notdb._capi.o
-L../../lib -lnotmuch -o
build/lib.linux-x86_64-3.7/notdb/_capi.abi3.so
On Tue, Oct 8, 2019 at 11:03 PM Floris Bruynooghe wrote:
>
> Hi all,
>
> IIRC there was a thread in August about another attempt at bringing
> the CFFI-based bindings on board as a Python3-only version. I
> believe there was a desire to re-name things but my searching-fu is
> failing me and I can no longer find the email thread.
>
> Anyway, I found the code, checked things work, updated tests on new
> python versions, added a very basic intergration with the test
> framework and squashed the commits. Otherwise the attached patch
> is just a plain dump of the current state so interested people have
>
