Re: [VOTE] Release PyLucene 3.6.0 rc2
Am 08.05.2012 02:20, schrieb Andi Vajda: > Please vote to release these artifacts as PyLucene 3.6.0-2. All tests are passing on Ubuntu 12.04 AMD64. This time I'm unable to test PyLucene 3.6 with our application since bobo browse is incompatible with Lucene 3.6. Here is my +1 Christian
Re: [VOTE] Release PyLucene 3.6.1-2
Am 21.08.2012 07:14, schrieb Andi Vajda: > Please vote to release these artifacts as PyLucene 3.6.1-2. -1, sorry :) test_ICUTransformFilter.py is segfaulting on my system. I haven't been able to get a sensible stack trace yet. I'm going to recompile everything with debug infos tomorrow. OS: Ubuntu 12.04 Arch: x86_64 Python: 2.7.3 (self compiled with UCS-2) ICU4C: 49.1.1 PyICU: 1.4 Christian
Re: Problems passing PyLucene objects to jcc-wrapped bobo-browse api
Adding the --import'ed .so libraries to the link line on Linux solve several of the problems you reported: - import order doesn't matter anymore - initVM() order doesn't matter anymore - class_ properties are now correct This is now checked into the branch_3x pylucene's branch. Please give it another try with your software, Thanks Andi! I'll give it a try tomorrow. I'll also ask Julian to provide some test and sample code for you. Christian
Re: Problems passing PyLucene objects to jcc-wrapped bobo-browse api
The rather strange bug is that the bobobrowse package doesn't contain any of the bobobrowse classes when I compile JCC, PyLucene and Bobo-Browse inside our environment. Let's see how it's behaving with your latest changes. Looks like this bug is gone for good. I suspect it was an incompatibility between my rpath and your dlopen() hack. Thanks! Christian
Re: [VOTE] Release PyLucene 2.9.3-1 and 3.0.2-1
Am 21.06.2010 17:00, schrieb Christian Heimes: >> With the recent - simultaneous - releases of Java Lucene 2.9.3 and 3.0.2, >> the PyLucene 2.9.3-1 and 3.0.2-1 releases closely tracking them are ready. > > I've upgrade PyLucene from 2.9.2 to 2.9.3 and JCC from 2.5.1 to 2.6. Our > application stack, unit tests and bobo-browse integration haven't shown > any issue. > > +1 from me I'm sorry to inform you, that I have to retract my +1. A full rebuild of JCC, PyLucene and Bobo has revealed an issue. I haven't noticed the issue before because I forgot to update to PyLucene 2.9.3 on my bobo-browse test box. The build error is a regression from PyLucene 2.9.2 to 2.9.3. python2.6 -m jcc.__main__ --shared --jar bobo-browse-2.5.0-rc1/dist/bobo-browse-2.5.0-rc1.jar --include bobo-browse-2.5.0-rc1/lib/master/servlet-api.jar --include bobo-browse-2.5.0-rc1/lib/master/spring-webmvc.jar --include bobo-browse-2.5.0-rc1/lib/master/spring.jar --include bobo-browse-2.5.0-rc1/lib/master/log4j.jar --include bobo-browse-2.5.0-rc1/lib/master/protobuf-java-2.2.0.jar --include bobo-browse-2.5.0-rc1/lib/master/xstream.jar --include bobo-browse-2.5.0-rc1/lib/master/fastutil.jar --include bobo-browse-2.5.0-rc1/lib/master/kamikaze-2.0.0.jar --classpath bobo-browse-2.5.0-rc1/lib/master/ant.jar --classpath bobo-browse-2.5.0-rc1/lib/master/xercesImpl.jar --classpath bobo-browse-2.5.0-rc1/lib/master/commons-collections.jar --classpath bobo-browse-2.5.0-rc1/lib/master/commons-cli.jar --classpath bobo-browse-2.5.0-rc1/lib/master/commons-configuration.jar --classpath bobo-browse-2.5.0-rc1/lib/master/commons-logging.jar --classpath bobo-browse-2.5.0-rc1/lib/master/commons-lang.jar --classpath bobo-browse-2.5.0-rc1/lib/master/commons-digester.jar --classpath bobo-browse-2.5.0-rc1/lib/master/commons-httpclient.jar --classpath bobo-browse-2.5.0-rc1/lib/master/zoie-2.0.0-rc1.jar --classpath bobo-browse-2.5.0-rc1/lib/master/json.jar --classpath bobo-browse-2.5.0-rc1/lib/master/dwr.jar --classpath bobo-browse-2.5.0-rc1/lib/master/xmlParserAPIs.jar --package java.util java.util.LinkedHashSet java.util.LinkedList --exclude com.browseengine.bobo.facets.FacetHandler\$TermCountSize --import lucene --python bobo --version 2.5.0-rc1 --files 2 --build Warning: renaming static method 'toString' on class com.browseengine.bobo.api.BrowseResult to 'toString_' since it is shadowed by non-static method of same name. running build_ext building 'bobo._bobo' extension creating build/temp.linux-x86_64-2.6 creating build/temp.linux-x86_64-2.6/build creating build/temp.linux-x86_64-2.6/build/_bobo gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -D_java_generics -DPYTHON=1 -D_jcc_shared=1 -I/usr/lib/jvm/java-6-sun/include -I/usr/lib/jvm/java-6-sun/include/linux -Ibuild/_bobo -I/usr/local/lib/python2.6/dist-packages/JCC-2.6-py2.6-linux-x86_64.egg/jcc/sources -I/usr/local/lib/python2.6/dist-packages/lucene-2.9.3-py2.6-linux-x86_64.egg/lucene/include -I/usr/include/python2.6 -c build/_bobo/functions.cpp -o build/temp.linux-x86_64-2.6/build/_bobo/functions.o -fno-strict-aliasing -Wno-write-strings cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++ In file included from /usr/include/python2.6/Python.h:8, from /usr/local/lib/python2.6/dist-packages/JCC-2.6-py2.6-linux-x86_64.egg/jcc/sources/JCCEnv.h:47, from /usr/local/lib/python2.6/dist-packages/JCC-2.6-py2.6-linux-x86_64.egg/jcc/sources/JObject.h:21, from /usr/local/lib/python2.6/dist-packages/lucene-2.9.3-py2.6-linux-x86_64.egg/lucene/include/java/lang/Object.h:4, from build/_bobo/functions.cpp:18: /usr/include/python2.6/pyconfig.h:1031:1: warning: "_POSIX_C_SOURCE" redefined In file included from /usr/include/stdio.h:28, from /usr/lib/jvm/java-6-sun/include/jni.h:21, from build/_bobo/functions.cpp:15: /usr/include/features.h:158:1: warning: this is the location of the previous definition In file included from /usr/include/python2.6/Python.h:8, from /usr/local/lib/python2.6/dist-packages/JCC-2.6-py2.6-linux-x86_64.egg/jcc/sources/JCCEnv.h:47, from /usr/local/lib/python2.6/dist-packages/JCC-2.6-py2.6-linux-x86_64.egg/jcc/sources/JObject.h:21, from /usr/local/lib/python2.6/dist-packages/lucene-2.9.3-py2.6-linux-x86_64.egg/lucene/include/java/lang/Object.h:4, from build/_bobo/functions.cpp:18: /usr/include/python2.6/pyconfig.h:1040:1: warning: "_XOPEN_SOURCE" redefined In file included from /usr/include/stdio.h:28, from /usr/lib/jvm/java-6-sun/include/jni.h:21, from build/_bobo/functions.cpp:15: /usr/include/features.h:160:1: warning: this is the location of the previous definition In file included from build/_bobo/funct
Re: [VOTE] Release PyLucene 2.9.3-1 and 3.0.2-1
Am 22.06.2010 17:42, schrieb Andi Vajda: > The issue has to do with the fact that PyLucene 2.9.3 is built with generics > support turned off (passing --no-generics to jcc) by default since Lucene > 2.9.3 makes no use of generics and is thus supported with Java 1.4 as well. > > If you --import lucene built with --no-generics you also must use > --no-generics when building bobo. > > You can also change PyLucene 2.9.3's Makefile to not use --no-generics or > switch to PyLucene 3.0.2 which uses Lucene 3.0.2, a version of Lucene that > requires Java 1.5 and uses generics. Ah! :) The thing is, bobo browse needs generic support for some features. Our test cases fail when I compile bobo browse without generics. Could you make the option --no-generics a make variable? Something like JCC_ARGS = "--no-generics" GENERATE=$(JCC) $(foreach jar,$(JARS),--jar $(jar)) \ $(JCC_ARGS) \ --package java.lang java.lang.System \ java.lang.Runtime \ ... This way I'm able to overwrite the option with a simple make argument instead of patching the Makefile. It might be wise to remove the default "--no-generics" to keep 2.9.3 compatible with 2.9.2. The option is a new feature. Christian
Re: [VOTE] Release PyLucene 2.9.3-1 and 3.0.2-1
> So I added 'JCCFLAGS?=--no-generics' to PyLucene's Makefile and uploaded new > release artifacts. Please check them out and let me know if all is working > as expected. Thanks a lot! It works like a charm. +1 again ;) Christian
Re: segfault in JCCEnv::deleteGlobalRef
Am 11.05.2011 18:56, schrieb Christian Heimes: > Am 11.05.2011 18:27, schrieb Andi Vajda: >> Does it crash as easily with Python 2.6 ? >> If not, then that could be an answer as to why this wasn't noticed before. > > With 20 test samples, it seems like Python 2.6 survives 50% longer than > Python 2.7. 100 samples: python2.6 cnt: 100, min: 0.886, max: 3.700, avg: 1.260 python2.7 cnt: 100, min: 0.888, max: 3.793, avg: 1.426
Re: setuptools not really setuptools on Ubuntu
Am 29.06.2011 18:13, schrieb Andi Vajda: > Sigh. The setuptools story is getting worse. I wonder how the 'distribute' > project is doing... It's the solution I used for the Python 3.1 jcc port I > did last summer. In particular, I wonder if they integrated my patch, for > that issue 43 I filed like four years ago. > > Do you know if there is a way to detect this special version of setuptools ? > If so, I could ensure the patch is applied if still needed. 'They' for setuptools is really just P.J. Eby. There hasn't been any serious development on setuptools in the past few years. Luckily Tarek has forked setuptools and started his work on distribute and distutils2. He is a very active developer and IMHO open to new ideas. Have you talked to him about the requirements for JCC? I'm sure he is going to integrate your patch soonish. There isn't a reason to support vanilla setuptools anymore once the patch is part of distribute. distribute is fully backward compatible to setuptoools. See http://pypi.python.org/pypi/distribute#about-the-fork for some background information. Christian
Re: PyLucene, multiprocessing, high thread count
Am 15.07.2011 10:10, schrieb Andi Vajda: > PyLucene embeds a Java VM. Thus, with each subprocess, a new JVM is created > with all its threads. This can get insane pretty quickly. The Java VM starts a lot of threads. On my Linux box eleven threads additional threads are running after initVM() has been called. >>> import lucene, os, psutil >>> psutil.Process(os.getpid()).get_num_threads() 1 >>> lucene.initVM() >>> psutil.Process(os.getpid()).get_num_threads() 12 Christian
Re: [VOTE] Release PyLucene 3.3 (rc3)
Am 21.07.2011 18:47, schrieb Andi Vajda: > Please vote to release these artifacts as PyLucene 3.3-3. +1 from me I've tested PyLucene on Linux (Ubuntu 10.04 as well as 11.04 on X86_64) with Python 2.6 and 2.7. My bobobrowse integration and the unit test suite of our applications are working, too. The grouping module is available, too. :) Christian
Re: [VOTE] Release PyLucene 3.5.0
Am 07.12.2011 03:39, schrieb Andi Vajda: > > The PyLucene 3.5.0-1 release closely tracking the recent release of > Apache Lucene 3.5.0 is ready. > > A release candidate is available from: > http://people.apache.org/~vajda/staging_area/ > > A list of changes in this release can be seen at: > http://svn.apache.org/repos/asf/lucene/pylucene/branches/pylucene_3_5/CHANGES > > PyLucene 3.5.0 is built with JCC 2.12 included in these release artifacts. Hello Andi, The CHANGES file doesn't mention JCC 2.12. Christian
Re: Feature request: include collation
Am 27.09.2010 03:59, schrieb Andi Vajda:
> I'm not sure at this point which should remain. There are advantages to
> both... I'm open to arguments in favor of either. You can see examples in
> the 3.x tree [3].
The 3.x branch doesn't build for me.
$ LC_ALL=C make ANT=ant JCC="python2.6 -m jcc.__main__ --shared"
NUM_FILES=2 PYTHON=python2.6
rm -f
lucene-java-3.x/contrib/icu/src/resources/org/apache/lucene/analysis/icu/utr30.dat
cd
lucene-java-3.x/contrib/icu/src/resources/org/apache/lucene/analysis/icu/;
/usr/sbin/icupkg --type l --add utr30.nrm new utr30.dat
udata_swap(): unknown data format 4e.72.6d.32 ("Nrm2")
icupkg: udata_swap(item 0) failed - U_UNSUPPORTED_ERROR
make: ***
[lucene-java-3.x/contrib/icu/src/resources/org/apache/lucene/analysis/icu/utr30.dat]
Error 16
I'm running Ubuntu 10.04 x64 with ICU 4.2.1.
Christian
Segfault during IndexWriter.addDocument()
I'm trying to port our application to PyLucene 3.x. Most changes were trivial but now I've hit a dead end. The test suite always seg faults when it tries to add a document to the index. writer.addDocument(document, analyzer) (Pdb) type(analyzer) (Pdb) type(analyzer).class_ (Pdb) type(document) (Pdb) type(document).class_ (Pdb) p writer (Pdb) n Exception RuntimeError: 'maximum recursion depth exceeded in __subclasscheck__' in ignored Exception RuntimeError: 'maximum recursion depth exceeded in __subclasscheck__' in ignored Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in ignored Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in ignored terminate called after throwing an instance of 'int' Aborted $ gdb /usr/bin/python2.6 core (gdb) bt #0 0x7fab2afd5a75 in *__GI_raise (sig=) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 #1 0x7fab2afd95c0 in *__GI_abort () at abort.c:92 #2 0x7fab231128e5 in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/libstdc++.so.6 #3 0x7fab23110d16 in ?? () from /usr/lib/libstdc++.so.6 #4 0x7fab23110d43 in std::terminate() () from /usr/lib/libstdc++.so.6 #5 0x7fab23110e3e in __cxa_throw () from /usr/lib/libstdc++.so.6 #6 0x7fab21944b1c in JCCEnv::reportException() const () from /usr/local/lib/python2.6/dist-packages/JCC-2.6-py2.6-linux-x86_64.egg/libjcc.so #7 0x7fab219446db in JCCEnv::callObjectMethod(_jobject*, _jmethodID*, ...) const () from /usr/local/lib/python2.6/dist-packages/JCC-2.6-py2.6-linux-x86_64.egg/libjcc.so #8 0x7fab21940c50 in JCCEnv::toString(_jobject*) const () from /usr/local/lib/python2.6/dist-packages/JCC-2.6-py2.6-linux-x86_64.egg/libjcc.so #9 0x7fab21eba0ce in t_JObject_str (self=0x6b853f0) at build/_lucene/JObject.cpp:145 #10 0x00454a81 in _PyObject_Str (v=) at ../Objects/object.c:425 #11 0x00454b3a in PyObject_Str (v=) at ../Objects/object.c:446 #12 0x0041f0c7 in PyObject_Call (func=, arg=, kw= ) at ../Objects/abstract.c:2492 #13 0x004a7378 in do_call (f= Frame 0x7fab0c874210, for file /usr/local/lib/python2.6/dist-packages/lucene-3.x-py2.6-linux-x86_64.egg/lucene/__init__.py, line 17, in __str__ (self=, writer=), throwflag=) at ../Python/ceval.c:3968 #14 call_function (f= Frame 0x7fab0c874210, for file /usr/local/lib/python2.6/dist-packages/lucene-3.x-py2.6-linux-x86_64.egg/lucene/__init__.py, line 17, in __str__ (self=, writer=), throwflag=) at ../Python/ceval.c:3773 #15 PyEval_EvalFrameEx (f= Frame 0x7fab0c874210, for file /usr/local/lib/python2.6/dist-packages/lucene-3.x-py2.6-linux-x86_64.egg/lucene/__init__.py, line 17, in __str__ (self=, writer=), throwflag=) at ../Python/ceval.c:2412 #16 0x004a9671 in PyEval_EvalCodeEx (co=0x41195d0, globals=, locals=, args=0x1, argcount=, kws=, kwcount=0, defs=0x0, defcount=0, closure=0x0) at ../Python/ceval.c:3000 #17 0x00537620 in function_call (func=, arg=(,), kw=0x0) at ../Objects/funcobject.c:524 #18 0x0041f0c7 in PyObject_Call (func=, arg=, kw=) at ../Objects/abstract.c:2492 #19 0x00427dff in instancemethod_call (func=, arg=(,), kw=0x0) at ../Objects/classobject.c:2579 #20 0x0041f0c7 in PyObject_Call (func=, arg=, kw= ) at ../Objects/abstract.c:2492 #21 0x004a1b03 in PyEval_CallObjectWithKeywords (func=, arg=(), kw=) at ../Python/ceval.c:3619 #22 0x0047796d in slot_tp_str (self=) at ../Objects/typeobject.c:5313 #23 0x00454a81 in _PyObject_Str (v=) at ../Objects/object.c:425 #24 0x00454b3a in PyObject_Str (v=) at ../Objects/object.c:446 #25 0x004c7f67 in PyErr_Display (exception='lucene', value=, tb=) at ../Python/pythonrun.c:1267 #26 0x004cf4ec in sys_excepthook (self=, args=) at ../Python/sysmodule.c:138 #27 0x0041f0c7 in PyObject_Call (func=, arg=, kw=) at ../Objects/abstract.c:2492 #28 0x004a1b03 in PyEval_CallObjectWithKeywords (func=, arg= (, , None), kw=) at ../Python/ceval.c:3619 #29 0x004c8db8 in PyErr_PrintEx (set_sys_last_vars=) at ../Python/pythonrun.c:1148 #30 0x7fab21940796 in _PythonException_getErrorInfo(JNIEnv_*, _jobject*) () from /usr/local/lib/python2.6/dist-packages/JCC-2.6-py2.6-linux-x86_64.egg/libjcc.so #31 0x7fab1ab67c48 in ?? () #32 0x7fab19a2e9f8 in ?? () #33 0x7fab026041b8 in ?? () #34 0x7fab119c4950 in ?? () #35 0x7fab02604218 in ?? () #36 0x7fab119c4e70 in ?? () #37 0x in ?? ()
Re: Segfault during IndexWriter.addDocument()
Am 27.09.2010 18:10, schrieb Christian Heimes: > I'm trying to port our application to PyLucene 3.x. Most changes were > trivial but now I've hit a dead end. The test suite always seg faults > when it tries to add a document to the index. Our subclass of lucene.PythonAnalyzer seems to trigger the unhandled java exception. I'll try to sort it out. Christian
Re: [VOTE] Release PyLucene 4.2.1-1
Am 18.04.2013 19:08, schrieb Andi Vajda: > > On Thu, 18 Apr 2013, Thomas Koch wrote: > >> Andi, >> I now get a different error while compiling __init__.cpp: >> >> org/apache/lucene/util/automaton/CompiledAutomaton$AUTOMATON_TYPE.h(42) : >> error C2059: Syntaxfehler: 'Zeichenfolge' >> org/apache/lucene/util/automaton/CompiledAutomaton$AUTOMATON_TYPE.h(42) : >> error C2238: Unerwartete(s) Token vor ';' >> >> The line complained about is #42 >> >> 40static CompiledAutomaton$AUTOMATON_TYPE *NONE; >> 41static CompiledAutomaton$AUTOMATON_TYPE *NORMAL; >> 42static CompiledAutomaton$AUTOMATON_TYPE *PREFIX; >> 43static CompiledAutomaton$AUTOMATON_TYPE *SINGLE; >> >> PREFIX seems to be another reserved word ... I could compile __init__.cpp >> after renaming PREFIX to PREFIX1. > > Instead of renaming PREFIX, could you please have JCC do it for you by > adding it to the list of reserved words in the JCC invocation via the > --reserved command line flag ? and rinse and repeat until all such conficts > due to macro definitions are solved ? > > Or were you able to complete the build already once PREFIX was renamed ? I'm pretty sure the Windows build issue is caused by the PREFIX macro in PC/pyconfig.h. I ran into the same issue a while ago. I have created a bug report for the issue http://bugs.python.org/issue17791 Christian
