Re: [VOTE] Release PyLucene 3.6.1-2
On Thu, 23 Aug 2012, Christian Heimes wrote: Am 21.08.2012 07:14, schrieb Andi Vajda: Please vote to release these artifacts as PyLucene 3.6.1-2. -1, sorry :) test_ICUTransformFilter.py is segfaulting on my system. I haven't been able to get a sensible stack trace yet. I'm going to recompile everything with debug infos tomorrow. OS: Ubuntu 12.04 Arch: x86_64 Python: 2.7.3 (self compiled with UCS-2) ICU4C: 49.1.1 PyICU: 1.4 Christian I investigated this and found that there are actually two problems here: - for test/test_ICUFoldingFilter.py, the version of ICU must be 4.8 with Lucene 3.x and 49 with Lucene 4.x (as explained by Robert earlier) - the segfault with test/test_ICUTransformFilter.py is due to a bug in PyICU 1.4: a UBool was passed to parseArgs where an int was expected, causing the stack to get corrupted I fixed the PyICU bug in rev 200 of its svn repository. I added a version check for test/ICUFoldingFilter.py to skip the test and emit a warning in case of an ICU_VERSION mismatch. I also got the ICU-based tests to work on 4.0 (see below). Andi.. $ ls test/test_ICU* | xargs -n 1 _install/bin/python . -- Ran 1 test in 0.009s OK .. -- Ran 2 tests in 0.009s OK -- Ran 4 tests in 0.199s OK
Re: [VOTE] Release PyLucene 3.6.1-2
On Mon, 20 Aug 2012, Andi Vajda wrote: The PyLucene 3.6.1-2 release tracking the (not so) recent release of Apache Lucene 3.6.1 is ready. A release candidate is available from: http://people.apache.org/~vajda/staging_area/ A list of changes in this release can be seen at: http://svn.apache.org/repos/asf/lucene/pylucene/branches/pylucene_3_6/CHANGES PyLucene 3.6.1 is built with JCC 2.14 included in these release artifacts: http://svn.apache.org/repos/asf/lucene/pylucene/trunk/jcc/CHANGES A list of Lucene Java changes can be seen at: http://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_3_6_1/lucene/CHANGES.txt Please vote to release these artifacts as PyLucene 3.6.1-2. Thanks ! Andi.. ps: the KEYS file for PyLucene release signing is at: http://svn.apache.org/repos/asf/lucene/pylucene/dist/KEYS http://people.apache.org/~vajda/staging_area/KEYS pps: here is my +1 This vote has passed. Thank you all who voted. Expect a release announcement shortly. Andi..
Re: [VOTE] Release PyLucene 3.6.1-2
On Aug 23, 2012, at 6:09, Christian Heimes wrote: > Am 23.08.2012 06:21, schrieb Andi Vajda: >> I'm getting errors there too (if I build PyICU into PyLucene's install >> tree, otherwise these tests are not enabled). I believe the problem lies >> with the utr30.dat resource file. Their format requirements probably >> changed with the upgrade to ICU 49. >> >> I remember having a lot of trouble reverse engineering what format these >> files should be transformed into when taken from the Java version, such as >> for example, resources/org/apache/lucene/analysis/icu/utr30.dat. >> >> As this feature is optional and rather experimental, I'm not prepared to >> block the release for that bug. Not that it doesn't need to be fixed but >> the tranformation of Lucene's Java ICU utr30.dat into C++ ICU 49's >> utr30.nrm via the /usr/local/sbin/icupkg utility needs to be figured out >> again for this to work properly. > > I concur with your analysis. The tests are passing when I use Ubuntu's > default version 48.1. > > With 49.1.1 test suite segfaults somehwere in > lucene/ICUTransformFilter.py. I'm unable to get the exact line number as > the values were optimized out by the compiler. > > Perhaps you could disable all ICU features when icu.ICU_VERSION is 49 or > newer? That would prohibit people from getting segfaults. I agree, a version check should be added to the tests and a proper warning should be emitted in case of mismatch. Is it worth respinning a release candidate for this or should I go ahead and proceed with the release since the vote now has received the required three PMC votes for release ? Andi.. > > Christian
Re: [VOTE] Release PyLucene 3.6.1-2
Am 23.08.2012 06:21, schrieb Andi Vajda: > I'm getting errors there too (if I build PyICU into PyLucene's install > tree, otherwise these tests are not enabled). I believe the problem lies > with the utr30.dat resource file. Their format requirements probably > changed with the upgrade to ICU 49. > > I remember having a lot of trouble reverse engineering what format these > files should be transformed into when taken from the Java version, such as > for example, resources/org/apache/lucene/analysis/icu/utr30.dat. > > As this feature is optional and rather experimental, I'm not prepared to > block the release for that bug. Not that it doesn't need to be fixed but > the tranformation of Lucene's Java ICU utr30.dat into C++ ICU 49's > utr30.nrm via the /usr/local/sbin/icupkg utility needs to be figured out > again for this to work properly. I concur with your analysis. The tests are passing when I use Ubuntu's default version 48.1. With 49.1.1 test suite segfaults somehwere in lucene/ICUTransformFilter.py. I'm unable to get the exact line number as the values were optimized out by the compiler. Perhaps you could disable all ICU features when icu.ICU_VERSION is 49 or newer? That would prohibit people from getting segfaults. Christian
Re: [VOTE] Release PyLucene 3.6.1-2
On Thu, Aug 23, 2012 at 12:21 AM, Andi Vajda wrote: > > On Thu, 23 Aug 2012, Christian Heimes wrote: > >> Am 21.08.2012 07:14, schrieb Andi Vajda: >>> >>> Please vote to release these artifacts as PyLucene 3.6.1-2. >> >> >> -1, sorry :) >> >> test_ICUTransformFilter.py is segfaulting on my system. I haven't been >> able to get a sensible stack trace yet. I'm going to recompile >> everything with debug infos tomorrow. > > > I'm getting errors there too (if I build PyICU into PyLucene's install tree, > otherwise these tests are not enabled). I believe the problem lies with the > utr30.dat resource file. Their format requirements probably changed with the > upgrade to ICU 49. > Hi Andi, you are correct. these files only work with 4.8.1.1 If you are interested in working with 49 (which uses a newer format version), then you have to either regenerate yourself or if you want you can just use our files from http://svn.apache.org/repos/asf/lucene/dev/branches/branch_4x/lucene/analysis/icu/src/resources/ since Steven Rowe already upgraded our 4.x branch to 49 and regenerated them for that. Of course if you do this you will break any users using 4.8.x as well unless you include both and conditionalize it based on the version: our code/resources only support the version we ship with (not arbitrary versions of ICU), so we don't worry about that. -- lucidworks.com
Re: [VOTE] Release PyLucene 3.6.1-2
On Thu, 23 Aug 2012, Christian Heimes wrote: Am 21.08.2012 07:14, schrieb Andi Vajda: Please vote to release these artifacts as PyLucene 3.6.1-2. -1, sorry :) test_ICUTransformFilter.py is segfaulting on my system. I haven't been able to get a sensible stack trace yet. I'm going to recompile everything with debug infos tomorrow. I'm getting errors there too (if I build PyICU into PyLucene's install tree, otherwise these tests are not enabled). I believe the problem lies with the utr30.dat resource file. Their format requirements probably changed with the upgrade to ICU 49. I remember having a lot of trouble reverse engineering what format these files should be transformed into when taken from the Java version, such as for example, resources/org/apache/lucene/analysis/icu/utr30.dat. As this feature is optional and rather experimental, I'm not prepared to block the release for that bug. Not that it doesn't need to be fixed but the tranformation of Lucene's Java ICU utr30.dat into C++ ICU 49's utr30.nrm via the /usr/local/sbin/icupkg utility needs to be figured out again for this to work properly. Andi.. OS: Ubuntu 12.04 Arch: x86_64 Python: 2.7.3 (self compiled with UCS-2) ICU4C: 49.1.1 PyICU: 1.4 Christian
Re: [VOTE] Release PyLucene 3.6.1-2
On Wed, Aug 22, 2012 at 3:40 PM, Andi Vajda wrote: > > Yes, the PyLucene site is a crying shame right now and it has been that way > for a year, ever since the switch to the CMS was made. I've had no time to > do a thorough cleanup there to get it into shape - and so has no one else in > the PyLucene user community. > > I view this CMS switch as an unfunded (timewise) mandate :-) > I'll see if i can fix the links... I don't really know what I am doing with the CMS either so no promises. -- lucidworks.com
Re: [VOTE] Release PyLucene 3.6.1-2
On Wed, 22 Aug 2012, Robert Muir wrote: On Tue, Aug 21, 2012 at 1:14 AM, Andi Vajda wrote: Please vote to release these artifacts as PyLucene 3.6.1-2. +1 (from Linux). some unrelated notes: something is up in my browser at least for the links to pylucene and jcc install instructions on http://lucene.apache.org/pylucene/ and http://lucene.apache.org/pylucene/install.html the direct link from jcc/INSTALL works fine (http://lucene.apache.org/pylucene/jcc/install.html) but it seems to have lost its CSS. Yes, the PyLucene site is a crying shame right now and it has been that way for a year, ever since the switch to the CMS was made. I've had no time to do a thorough cleanup there to get it into shape - and so has no one else in the PyLucene user community. I view this CMS switch as an unfunded (timewise) mandate :-) I noticed this because i tried on a new linux system where i didnt have setuptools and was trying to get shared mode working (applied patch etc), ultimately I just ended out testing with shared mode disabled (which worked fine). It's good to know this configuration is also still working. Thank you for your vote ! Andi..
Re: [VOTE] Release PyLucene 3.6.1-2
On Tue, Aug 21, 2012 at 1:14 AM, Andi Vajda wrote: > > Please vote to release these artifacts as PyLucene 3.6.1-2. > +1 (from Linux). some unrelated notes: something is up in my browser at least for the links to pylucene and jcc install instructions on http://lucene.apache.org/pylucene/ and http://lucene.apache.org/pylucene/install.html the direct link from jcc/INSTALL works fine (http://lucene.apache.org/pylucene/jcc/install.html) but it seems to have lost its CSS. I noticed this because i tried on a new linux system where i didnt have setuptools and was trying to get shared mode working (applied patch etc), ultimately I just ended out testing with shared mode disabled (which worked fine). -- lucidworks.com
Re: [VOTE] Release PyLucene 3.6.1-2
+1 I tested on OS X 10.6.8, indexing first 100K docs from Wikipedia and running a few searches. Mike McCandless http://blog.mikemccandless.com On Tue, Aug 21, 2012 at 3:37 AM, Thomas Koch wrote: > Thanks, JCC 2.14 now builds just fine with MS-VC9 on Windows. Could also > build pylucene 3.6.1 and run the tests (except of the usual suspects on > windows that fail due to the file lock issue). > > So +1 for release. > > kind regards > Thomas > -- > OrbiTeam Software GmbH & Co. KG > http://www.orbiteam.de > >> -Ursprüngliche Nachricht- >> Von: Andi Vajda [mailto:va...@apache.org] >> Gesendet: Dienstag, 21. August 2012 07:14 >> An: pylucene-dev@lucene.apache.org >> Cc: gene...@lucene.apache.org >> Betreff: [VOTE] Release PyLucene 3.6.1-2 >> >> >> The PyLucene 3.6.1-2 release tracking the (not so) recent release of > Apache >> Lucene 3.6.1 is ready. >> >> A release candidate is available from: >> http://people.apache.org/~vajda/staging_area/ >> >> A list of changes in this release can be seen at: >> http://svn.apache.org/repos/asf/lucene/pylucene/branches/pylucene_3_6/ >> CHANGES >> >> PyLucene 3.6.1 is built with JCC 2.14 included in these release artifacts: >> http://svn.apache.org/repos/asf/lucene/pylucene/trunk/jcc/CHANGES >> >> A list of Lucene Java changes can be seen at: >> http://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_3_6_1/lucen >> e/CHANGES.txt >> >> Please vote to release these artifacts as PyLucene 3.6.1-2. >> >> Thanks ! >> >> Andi.. >> >> ps: the KEYS file for PyLucene release signing is at: >> http://svn.apache.org/repos/asf/lucene/pylucene/dist/KEYS >> http://people.apache.org/~vajda/staging_area/KEYS >> >> pps: here is my +1 > >