Test failures with pylucene-2.4.1-1
Hi,
I built pylucene-2.4.1-1 on Fedora 14.
( https://tuxdna.wordpress.com/2011/02/25/pylucene-on-fedora-14/ )
When I did make test, I found test failures which I am pasting below:
===BEGIN===
find test -name 'test_*.py' | xargs -t -n 1 /usr/bin/python
/usr/bin/python test/test_RewriteQuery.py
.
--
Ran 1 test in 0.050s
OK
/usr/bin/python test/test_Highlighter.py
..
--
Ran 2 tests in 0.115s
OK
Searching for: wicked
A bwicked/b problem... of the problem. bWicked/b
Searching for: wicked
A Bwicked/B problem is one for which each... the understanding of
the problem. BWicked/B
/usr/bin/python test/test_PositionIncrement.py
..
--
Ran 2 tests in 0.096s
OK
/usr/bin/python test/test_PythonQueryParser.py
..
--
Ran 2 tests in 0.009s
OK
/usr/bin/python test/test_FilteredQuery.py
..
--
Ran 2 tests in 0.110s
OK
/usr/bin/python test/test_FuzzyQuery.py
..
--
Ran 2 tests in 0.105s
OK
/usr/bin/python test/test_Binary.py
E
==
ERROR: testBinary (__main__.BinaryTestCase)
--
Traceback (most recent call last):
File test/test_Binary.py, line 32, in testBinary
83, 89, 105, 56, 95, 75, 0, 0, 14, 215, 128])
File test/test_Binary.py, line 27, in binary
assert c == v and b == [ord(a) for a in v]
TypeError: ord() expected string of length 1, but int found
--
Ran 1 test in 0.002s
FAILED (errors=1)
/usr/bin/python test/test_bug1763.py
.
--
Ran 1 test in 0.106s
OK
/usr/bin/python test/test_StopAnalyzer.py
..
--
Ran 2 tests in 0.001s
OK
/usr/bin/python test/test_PrefixFilter.py
.
--
Ran 1 test in 0.095s
OK
/usr/bin/python test/test_bug1842.py
.
--
Ran 1 test in 0.099s
OK
/usr/bin/python test/test_PrefixQuery.py
.
--
Ran 1 test in 0.097s
OK
/usr/bin/python test/test_RangeQuery.py
..
--
Ran 2 tests in 0.114s
OK
/usr/bin/python test/test_PythonDirectory.py
ETesting Indexing Incremental Looping
indexing 0
E
==
ERROR: test_FieldEnumeration (__main__.PythonDirectoryTests)
--
Traceback (most recent call last):
File /home/sansari/Downloads/pylucene-2.4.1-1/test/test_PyLucene.py,
line 212, in test_FieldEnumeration
self.test_indexDocument()
File /home/sansari/Downloads/pylucene-2.4.1-1/test/test_PyLucene.py,
line 62, in test_indexDocument
self.closeStore(store, writer)
File test/test_PythonDirectory.py, line 222, in closeStore
store.close()
File test/test_PythonDirectory.py, line 144, in close
stream.close()
File test/test_PythonDirectory.py, line 116, in close
super(PythonFileStreamOutput, self).close()
JavaError: org.apache.jcc.PythonException: sequence item 0: expected
string, int found
Traceback (most recent call last):
File test/test_PythonDirectory.py, line 127, in flushBuffer
self.fh.write(''.join(buffer))
TypeError: sequence item 0: expected string, int found
Java stacktrace:
org.apache.jcc.PythonException: sequence item 0: expected string, int found
Traceback (most recent call last):
File test/test_PythonDirectory.py, line 127, in flushBuffer
self.fh.write(''.join(buffer))
TypeError: sequence item 0: expected string, int found
at org.apache.pylucene.store.PythonIndexOutput.flushBuffer(Native
Method)
at org.apache.pylucene.store.PythonIndexOutput.flushBuffer(Unknown
Source)
at
org.apache.lucene.store.BufferedIndexOutput.flushBuffer(BufferedIndexOutput.java:96)
at
org.apache.lucene.store.BufferedIndexOutput.flush(BufferedIndexOutput.java:85)
at
org.apache.lucene.store.BufferedIndexOutput.close(BufferedIndexOutput.java:109)
==
ERROR: test_IncrementalLoop (__main__.PythonDirectoryTests)
--
Traceback (most recent call last):
File test/test_PythonDirectory.py, line 229, in
Re: pass compressed string
Hi Andi,
Thanks, the JArray_byte() does what I needed - I was (wrongly) passing
bytestring (which I think got automatically converted to unicode) and
trying to get bytes of that string was not correct.
Though it would be interesting to find out if it is possible to pass
string and get the bytes in java, I don't know if what conversion
happening on the jni side, or only in java - i shall do some reading
Example in python:
In [4]: s = zlib.compress(python)
In [5]: repr(s)
Out[5]: 'x\\x9c+\\xa8,\\xc9\\xc8\\xcf\\x03\\x00\\tW\\x02\\xa3'
In [6]: lucene.JArray_byte(s)
Out[6]: JArraybyte(120, -100, 43, -88, 44, -55, -56, -49, 3, 0, 9, 87, 2, -93)
The same thing in Jython:
s = zlib.compress(python)
s
'x\x9c+\xa8,\xc9\xc8\xcf\x03\x00\tW\x02\xa3'
repr(s)
'x\\x9c+\\xa8,\\xc9\\xc8\\xcf\\x03\\x00\\tW\\x02\\xa3'
String(s).getBytes()
array('b', [120, -62, -100, 43, -62, -88, 44, -61, -119, -61, -120,
-61, -113, 3, 0, 9, 87, 2, -62, -93])
String(s).getBytes('utf8')
array('b', [120, -62, -100, 43, -62, -88, 44, -61, -119, -61, -120,
-61, -113, 3, 0, 9, 87, 2, -62, -93])
String(s).getBytes('utf16')
array('b', [-2, -1, 0, 120, 0, -100, 0, 43, 0, -88, 0, 44, 0, -55, 0,
-56, 0, -49, 0, 3, 0, 0, 0, 9, 0, 87, 0, 2, 0, -93])
String(s).getBytes('ascii')
array('b', [120, 63, 43, 63, 44, 63, 63, 63, 3, 0, 9, 87, 2, 63])
Roman
On Thu, Feb 24, 2011 at 3:42 AM, Andi Vajda va...@apache.org wrote:
On Thu, 24 Feb 2011, Roman Chyla wrote:
I would like to transfer results from python to java:
hello = zlib.compress(hello)
on the java side do:
byte[] data = string.getBytes()
But I am not successful. Is there any translation going on somewhere?
Can you be more specific ?
Actual lines of code, errors, expected results, actual results...
An array of bytes in JCC is not created with a string but a
JArray('byte')(len or str)
import lucene
lucene.initVM()
jcc.JCCEnv object at 0x1004100d8
lucene.JArray('byte')(10)
JArraybyte(0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
lucene.JArray('byte')(abcd)
JArraybyte(97, 98, 99, 100)
Andi..
Re: Test failures with pylucene-2.4.1-1
On Feb 25, 2011, at 4:34, tuxdna tux...@gmail.com wrote:
Hi,
I built pylucene-2.4.1-1 on Fedora 14.
( https://tuxdna.wordpress.com/2011/02/25/pylucene-on-fedora-14/ )
When I did make test, I found test failures which I am pasting below:
This looks like a version of jcc too new for this old a pylucene is
used.
Which version of jcc did you use ?
The one that came with pylucene 2.4.1 or a newer one ?
Andi..
===BEGIN===
find test -name 'test_*.py' | xargs -t -n 1 /usr/bin/python
/usr/bin/python test/test_RewriteQuery.py
.
--
Ran 1 test in 0.050s
OK
/usr/bin/python test/test_Highlighter.py
..
--
Ran 2 tests in 0.115s
OK
Searching for: wicked
A bwicked/b problem... of the problem. bWicked/b
Searching for: wicked
A Bwicked/B problem is one for which each... the
understanding of
the problem. BWicked/B
/usr/bin/python test/test_PositionIncrement.py
..
--
Ran 2 tests in 0.096s
OK
/usr/bin/python test/test_PythonQueryParser.py
..
--
Ran 2 tests in 0.009s
OK
/usr/bin/python test/test_FilteredQuery.py
..
--
Ran 2 tests in 0.110s
OK
/usr/bin/python test/test_FuzzyQuery.py
..
--
Ran 2 tests in 0.105s
OK
/usr/bin/python test/test_Binary.py
E
==
ERROR: testBinary (__main__.BinaryTestCase)
--
Traceback (most recent call last):
File test/test_Binary.py, line 32, in testBinary
83, 89, 105, 56, 95, 75, 0, 0, 14, 215, 128])
File test/test_Binary.py, line 27, in binary
assert c == v and b == [ord(a) for a in v]
TypeError: ord() expected string of length 1, but int found
--
Ran 1 test in 0.002s
FAILED (errors=1)
/usr/bin/python test/test_bug1763.py
.
--
Ran 1 test in 0.106s
OK
/usr/bin/python test/test_StopAnalyzer.py
..
--
Ran 2 tests in 0.001s
OK
/usr/bin/python test/test_PrefixFilter.py
.
--
Ran 1 test in 0.095s
OK
/usr/bin/python test/test_bug1842.py
.
--
Ran 1 test in 0.099s
OK
/usr/bin/python test/test_PrefixQuery.py
.
--
Ran 1 test in 0.097s
OK
/usr/bin/python test/test_RangeQuery.py
..
--
Ran 2 tests in 0.114s
OK
/usr/bin/python test/test_PythonDirectory.py
ETesting Indexing Incremental Looping
indexing 0
E
==
ERROR: test_FieldEnumeration (__main__.PythonDirectoryTests)
--
Traceback (most recent call last):
File /home/sansari/Downloads/pylucene-2.4.1-1/test/
test_PyLucene.py,
line 212, in test_FieldEnumeration
self.test_indexDocument()
File /home/sansari/Downloads/pylucene-2.4.1-1/test/
test_PyLucene.py,
line 62, in test_indexDocument
self.closeStore(store, writer)
File test/test_PythonDirectory.py, line 222, in closeStore
store.close()
File test/test_PythonDirectory.py, line 144, in close
stream.close()
File test/test_PythonDirectory.py, line 116, in close
super(PythonFileStreamOutput, self).close()
JavaError: org.apache.jcc.PythonException: sequence item 0: expected
string, int found
Traceback (most recent call last):
File test/test_PythonDirectory.py, line 127, in flushBuffer
self.fh.write(''.join(buffer))
TypeError: sequence item 0: expected string, int found
Java stacktrace:
org.apache.jcc.PythonException: sequence item 0: expected string,
int found
Traceback (most recent call last):
File test/test_PythonDirectory.py, line 127, in flushBuffer
self.fh.write(''.join(buffer))
TypeError: sequence item 0: expected string, int found
at org.apache.pylucene.store.PythonIndexOutput.flushBuffer(Native
Method)
at org.apache.pylucene.store.PythonIndexOutput.flushBuffer
(Unknown Source)
at org.apache.lucene.store.BufferedIndexOutput.flushBuffer
(BufferedIndexOutput.java:96)
at org.apache.lucene.store.BufferedIndexOutput.flush
(BufferedIndexOutput.java:85)
at org.apache.lucene.store.BufferedIndexOutput.close
(BufferedIndexOutput.java:109)
==
ERROR:
Re: Test failures with pylucene-2.4.1-1
I built pylucene-2.4.1-1 on Fedora 14. ( https://tuxdna.wordpress.com/2011/02/25/pylucene-on-fedora-14/ ) When I did make test, I found test failures which I am pasting below: This looks like a version of jcc too new for this old a pylucene is used. Which version of jcc did you use ? The one that came with pylucene 2.4.1 or a newer one ? I installed it using pip-python, so it would be the latest available there: $ pip-python search jcc JCC - a C++ code generator for calling Java from C++/Python INSTALLED: 2.7 (latest) The one with pylucene 2.4.1 is JCC 2.2 Then I followed this link http://lucene.apache.org/pylucene/jcc/documentation/install.html#shared and the jcc/setup.py file which says that setuptools need to be atleast 0.6c7 for linux. I have python-setuptools 0.6.14 on my system. Then checked that sh_link_shared_object from setuptools.command.build_ext import sh_link_shared_object By I was sure that setuptools support shared linking. JCC 2.2 built fine and I installed it. However when I built the pylucene 2.4.1, I got this this error:' ===BEGIN=== $ make ... ... /usr/bin/python -m jcc --shared --jar lucene-java-2.4.1/build/lucene-core-2.4.1.jar --jar lucene-java-2.4.1/build/contrib/snowball/lucene-snowball-2.4.1.jar --jar lucene-java-2.4.1/build/contrib/highlighter/lucene-highlighter-2.4.1.jar --jar lucene-java-2.4.1/build/contrib/analyzers/lucene-analyzers-2.4.1.jar --jar lucene-java-2.4.1/build/contrib/regex/lucene-regex-2.4.1.jar --jar lucene-java-2.4.1/build/contrib/queries/lucene-queries-2.4.1.jar --jar lucene-java-2.4.1/build/contrib/instantiated/lucene-instantiated-2.4.1.jar --jar build/jar/extensions.jar --package java.lang java.lang.System java.lang.Runtime --package java.util java.util.Arrays java.text.SimpleDateFormat --package java.io java.io.StringReader java.io.InputStreamReader java.io.FileInputStream --exclude org.apache.lucene.queryParser.Token --exclude org.apache.lucene.queryParser.TokenMgrError --exclude org.apache.lucene.queryParser.QueryParserTokenManager --exclude org.apache.lucene.queryParser.ParseException --python lucene --mapping org.apache.lucene.document.Document 'get:(Ljava/lang/String;)Ljava/lang/String;' --mapping java.util.Properties 'getProperty:(Ljava/lang/String;)Ljava/lang/String;' --sequence org.apache.lucene.search.Hits 'length:()I' 'doc:(I)Lorg/apache/lucene/document/Document;' --version 2.4.1 --module python/collections.py --files 2 --build Warning: renaming static method 'isInfinite' on class java.lang.Double to 'isInfinite_' since it is shadowed by non-static method of same name. Warning: renaming static method 'isNaN' on class java.lang.Double to 'isNaN_' since it is shadowed by non-static method of same name. Warning: renaming static method 'toString' on class java.lang.Double to 'toString_' since it is shadowed by non-static method of same name. Warning: renaming static method 'toString' on class java.lang.Integer to 'toString_' since it is shadowed by non-static method of same name. Warning: renaming static method 'fileModified' on class org.apache.lucene.store.FSDirectory to 'fileModified_' since it is shadowed by non-static method of same name. Warning: renaming static method 'toString' on class java.lang.Long to 'toString_' since it is shadowed by non-static method of same name. Warning: renaming static method 'toString' on class java.lang.Boolean to 'toString_' since it is shadowed by non-static method of same name. Warning: renaming static method 'toString' on class java.lang.Character to 'toString_' since it is shadowed by non-static method of same name. Traceback (most recent call last): File /usr/lib64/python2.7/runpy.py, line 162, in _run_module_as_main __main__, fname, loader, pkg_name) File /usr/lib64/python2.7/runpy.py, line 72, in _run_code exec code in run_globals File /usr/lib64/python2.7/site-packages/JCC-2.2-py2.7-linux-x86_64.egg/jcc/__main__.py, line 5, in module cpp.jcc(sys.argv) File /usr/lib64/python2.7/site-packages/JCC-2.2-py2.7-linux-x86_64.egg/jcc/cpp.py, line 477, in jcc shared, compiler, modules) File /usr/lib64/python2.7/site-packages/JCC-2.2-py2.7-linux-x86_64.egg/jcc/python.py, line 1289, in compile raise NotImplementedError, JCC was not built with --shared mode support, see JCC's INSTALL file for more information NotImplementedError: JCC was not built with --shared mode support, see JCC's INSTALL file for more information make: *** [compile] Error 255 ===END=== Please tell me what am I doing wrong here? Thanks. /tuxdna
