Hi,
Christian Heimes, Dirk Rothe, and I have jcc-wrapped bobo-browse
(http://code.google.com/p/bobo-browse/) in order to add faceted search
capabilities to PyLucene. However, the two modules don't play well together, as
wrappers from PyLucene cannot be used in a bobo-browse context and vice versa.
What is the best way to get classes from two different jcc python extension
modules to interact?
Setup:
X64 Ubuntu 9.10 Python 2.6 Tried both versions:
JCC-2.4.1-py2.6-linux-x86_64.egg compiled with --shared
JCC-2.5.1-py2.6-linux-x86_64.egg compiled with --shared
Tried both versions:
lucene-2.9.1-py2.6-linux-x86_64.egg
lucene-2.9.2-py2.6-linux-x86_64.egg
bobobrowse-2.5.0_rc1-py2.6-linux-x86_64.egg
Both the bobo-browse python extension module and PyLucene are attached to the
main python thread and have been initialized using their respective initVM
methods.
I have attached the makefile (modified from PyLucene) that was used to build
bobo-browse.
#---- START CODE ----
import bobobrowse as bb
import lucene
import os
CLASSPATH = os.pathsep.join((lucene.CLASSPATH, bb.CLASSPATH)
lucene.initVM(CLASSPATH)
bb.initVM(CLASSPATH)
typeHandler = bb.MultiValueFacetHandler("type")
placeHandler = bb.MultiValueFacetHandler("id")
facetHandlers = lucene.JArray('object')(2, bb.FacetHandler)
facetHandlers[0] = typeHandler
facetHandlers[1] = placeHandler
reader = bb.IndexReader.open('/path/to/my/lucene/index', True)
facetHandlers = bb.Arrays.asList(facetHandlers)
indexReader = bb.BoboIndexReader.getInstance(reader, facetHandlers)
#---- END CODE ----
Expected:
The example code above works - java objects from classes wrapped as part of the
PyLucene module can transparently be used by other python extension modules
such as bobo-browse.
Experienced:
Throws Errors:
Traceback (most recent call last):
File "./dynamic/vls/search/facets.py", line 115, in <module>
test()
File "./dynamic/smc/jcc/__init__.py", line 52, in wrapper
return func(*args, **kwargs)
File "./dynamic/vls/search/facets.py", line 57, in test
facetHandlers = lucene.JArray('object')(2, bb.FacetHandler)
ValueError: <type 'FacetHandler'>
and
bobobrowse.InvalidArgsError: (<type 'BoboIndexReader'>, 'getInstance', (<IndexReader:
org.apache.lucene.index.directoryowningrea...@30419d05>, <List:
[com.browseengine.bobo.facets.impl.multivaluefacethand...@3f829e6f,
com.browseengine.bobo.facets.impl.multivaluefacethand...@2980f96c]>))
respectively.
The Errors can be avoided by changing the corresponding lines in the example
above to:
facetHandlers = bb.JArray('object')(2, bb.FacetHandler)
reader = bb.IndexReader.open('/path/to/my/lucene/index', True)
However, this only works because lucene's IndexReader is referenced by
bobo-browse and has therefore been wrapped at build-time. At later stages other
lucene classes are needed to work with bobobrowse that no wrappers have been
built for (e.g. QueryParser). I understand that jcc can be forced to built and
include wrappers for e.g. org.apache.lucene.queryParser.QueryParser that are
then accessible for use with classes from the bobo-browse module as e.g
'bobobrowse.QueryParser'. The most simple solution is calling jcc with an
additional --jar lucene.jar parameter - in effect leading to lucene being
installed twice: First self-contained PyLucene module and, second, as part of
bobo-browse in the flat bobo-browse module namespace. This is not quite
desirable:
Since
bb.IndexReader.class_ == lucene.IndexReader.class_
--> class org.apache.lucene.index.IndexReader, anyways, it would arguably be
more intuitive to be able to use the available wrappers in the context of other
jcc modules even if they have not specifically been forced into that other module.
Is there any way to obtain the desired behavior?
Thank you very much and all the best,
Julian Maibaum
j.maibaum(at)semantics.de
s<e>mantics GmbH
http://www.semantics.de
# Makefile for building bobo-browser bindings for Python
#
# Based on PyLucene's Makefile
VERSION=2.5.0-rc1
BOBO_SVN_VER=HEAD
BOBO_VER=2.5.0-rc1
BOBO_SVN=http://bobo-browse.googlecode.com/svn/trunk
PYBOBO:=$(shell pwd)
BOBO_BROWSE=bobo-browse-$(BOBO_VER)
BOBO_BROWSE_JAR=$(BOBO_BROWSE)/dist/bobo-browse-$(BOBO_VER).jar
ANT_OPTS=
#ANT_OPTS=-Dhttp.proxyHost=192.168.1.1 -Dhttp.proxyPort=3128
ANT=ANT_OPTS="${ANT_OPTS}" ant
PYTHON=python2.6
JCC=$(PYTHON) -m jcc.__main__ --shared
NUM_FILES=2
ifeq ($(DEBUG),1)
DEBUG_OPT=--debug
endif
.PHONY: generate compile install default all clean realclean \
sources test jars distrib
default: all
$(BOBO_BROWSE):
svn co -r $(BOBO_SVN_VER) $(BOBO_SVN) $(BOBO_BROWSE)
sources: $(BOBO_BROWSE)
#to-orig: sources
# mkdir -p $(LUCENE)-orig
# tar -C $(LUCENE) -cf - . | tar -C $(LUCENE)-orig -xvf -
#from-orig: $(LUCENE)-orig
# mkdir -p $(LUCENE)
# tar -C $(LUCENE)-orig -cf - . | tar -C $(LUCENE) -xvf -
bobobrowse:
rm -f $(BOBO_BROWSE_JAR)
$(MAKE) $(BOBO_BROWSE_JAR)
$(BOBO_BROWSE_JAR): $(BOBO_BROWSE)
cd $(BOBO_BROWSE); $(ANT) -Dversion=$(BOBO_VER) jars
JARS=$(BOBO_BROWSE_JAR)
INCLUDES= \
$(BOBO_BROWSE)/lib/master/servlet-api.jar \
$(BOBO_BROWSE)/lib/master/json.jar \
$(BOBO_BROWSE)/lib/master/commons-configuration.jar \
$(BOBO_BROWSE)/lib/master/dwr.jar \
$(BOBO_BROWSE)/lib/master/commons-logging.jar \
$(BOBO_BROWSE)/lib/master/fastutil.jar \
$(BOBO_BROWSE)/lib/master/log4j.jar \
$(BOBO_BROWSE)/lib/master/commons-lang.jar \
$(BOBO_BROWSE)/lib/master/commons-digester.jar \
$(BOBO_BROWSE)/lib/master/protobuf-java-2.2.0.jar \
$(BOBO_BROWSE)/lib/master/xmlParserAPIs.jar \
$(BOBO_BROWSE)/lib/master/xstream.jar \
$(BOBO_BROWSE)/lib/master/kamikaze-2.0.0.jar \
$(BOBO_BROWSE)/lib/master/commons-httpclient.jar \
$(BOBO_BROWSE)/lib/master/zoie-2.0.0-rc1.jar \
$(BOBO_BROWSE)/lib/master/spring-webmvc.jar \
$(BOBO_BROWSE)/lib/master/spring.jar \
$(BOBO_BROWSE)/lib/master/commons-collections.jar \
$(BOBO_BROWSE)/lib/master/commons-cli.jar \
CLASSPATHS= \
$(BOBO_BROWSE)/lib/master/lucene-core.jar \
$(BOBO_BROWSE)/lib/master/ant.jar \
$(BOBO_BROWSE)/lib/master/lucene-highlighter.jar \
$(BOBO_BROWSE)/lib/master/xercesImpl.jar \
jars: $(JARS)
GENERATE=$(JCC) $(foreach jar,$(JARS),--jar $(jar)) \
$(foreach jar,$(INCLUDES),--include $(jar)) \
$(foreach jar,$(CLASSPATHS),--classpath $(jar)) \
--package java.lang java.lang.System \
java.lang.Runtime \
--package java.util \
java.util.Arrays \
java.text.SimpleDateFormat \
--package java.io java.io.StringReader \
java.io.InputStreamReader \
java.io.FileInputStream \
--python bobobrowse \
--version $(BOBO_VER) \
--files $(NUM_FILES)
generate: jars
$(GENERATE)
compile: jars
$(GENERATE) --build $(DEBUG_OPT)
install: jars
$(GENERATE) --install $(DEBUG_OPT) $(INSTALL_OPT)
bdist: jars
$(GENERATE) --bdist
all: sources jars compile
@echo build of bobo-browse complete
clean:
if test -f $(BOBO_BROWSE)/build.xml; then cd $(BOBO_BROWSE); $(ANT)
clean; fi
rm -rf build
realclean:
rm -rf $(BOBO_BROWSE)
rm -rf build dist
BUILD_TEST:=$(PYBOBO)/build/test
ifeq ($(findstring CYGWIN,$(shell uname)),CYGWIN)
BUILD_TEST:=`cygpath -aw $(BUILD_TEST)`
endif
install-test:
mkdir -p $(BUILD_TEST)
PYTHONPATH=$(BUILD_TEST) $(GENERATE) --install $(DEBUG_OPT)
--install-dir $(BUILD_TEST)
test-clean:
rm -rf $(BUILD_TEST)
test: install-test
$(PYTHON) test.py
ARCHIVE=bobo_browse-$(VERSION)-src.tar.gz
#distrib:
# mkdir -p distrib
# svn export . distrib/pylucene-$(VERSION)
# tar -cf - --exclude build $(LUCENE) | tar -C
distrib/pylucene-$(VERSION) -xvf -
# mkdir distrib/pylucene-$(VERSION)/doc
# tar -C $(SITE) -cf - . | tar -C distrib/pylucene-$(VERSION)/doc -xvf -
# cd distrib; tar -cvzf $(ARCHIVE) pylucene-$(VERSION)
# cd distrib; gpg2 --armor --output $(ARCHIVE).asc --detach-sig $(ARCHIVE)
# cd distrib; openssl md5 < $(ARCHIVE) > $(ARCHIVE).md5
#
#stage:
# cd distrib; scp -p $(ARCHIVE) $(ARCHIVE).asc $(ARCHIVE).md5 \
# people.apache.org:public_html/staging_area
#
#release:
# cd distrib; scp -p $(ARCHIVE) $(ARCHIVE).asc $(ARCHIVE).md5 \
people.apache.org:/www/www.apache.org/dist/lucene/pylucene
print-%:
@echo $* = $($*)
