I tried to reach the autors twice, but with no luck. I've seen some posts where people finally were able to lunch it (without much pain). I don't know. If any pro would be so nice to try to run the stempel on his/her machine and paste me some verbose step by step solution I would really appreciate.
Cheers, Jakub Godawa. 2010/11/13 Lance Norskog <goks...@gmail.com>: > I don't know of the Stempel jar includes the Java source. At this point I > think you should ask the author to Stempel to make a Solr front-end for it. > It's very simple for him. > > Jakub Godawa wrote: >> >> Am I not doing it in the point no 4? I am compiling all the folder >> that was extracted before, but now with that new class file. >> >> 2010/11/12 Lance Norskog<goks...@gmail.com>: >> >>> >>> I think you have to compile all of the stempel source including your >>> filter factory into one jar at the same time. Everybody does this; I >>> don't know how different Java versions make class file binaries. >>> >>> On Thu, Nov 11, 2010 at 3:06 AM, Jakub Godawa<jakub.god...@gmail.com> >>> wrote: >>> >>>> >>>> Hi! Sorry for such a break, but I was moving house... anyway: >>>> >>>> 1. I took the >>>> ~/apache-solr/src/java/org/apache/solr/analysis/StandardFilterFactory.java >>>> file and modified it (named as StempelFilterFactory.java) in Vim that >>>> way: >>>> >>>> package org.getopt.solr.analysis; >>>> >>>> import org.apache.lucene.analysis.TokenStream; >>>> import org.apache.lucene.analysis.standard.StandardFilter; >>>> >>>> public class StempelTokenFilterFactory extends BaseTokenFilterFactory { >>>> public StempelFilter create(TokenStream input) { >>>> return new StempelFilter(input); >>>> } >>>> } >>>> >>>> 2. Then I put the file to the extracted stempel-1.0.jar in >>>> ./org/getopt/solr/analysis/ >>>> 3. Then I created a class from it: jar -cf >>>> StempelTokenFilterFactory.class StempelFilterFactory.java >>>> 4. Then I created new stempel-1.0.jar archive: jar -cf stempel-1.0.jar >>>> -C ./stempel-1.0/ . >>>> 5. Then in schema.xml I've put: >>>> >>>> <fieldType name="text_pl" class="solr.TextField"> >>>> <analyzer> >>>> <tokenizer class="solr.WhitespaceTokenizerFactory"/> >>>> <filter class="solr.LowerCaseFilterFactory"/> >>>> <filter >>>> class="org.getopt.solr.analysis.StempelTokenFilterFactory" /> >>>> </analyzer> >>>> </fieldType> >>>> >>>> 6. I started the solr server and I recieved the following error: >>>> >>>> 2010-11-11 11:50:56 org.apache.solr.common.SolrException log >>>> SEVERE: java.lang.ClassFormatError: Incompatible magic value >>>> 1347093252 in class file >>>> org/getopt/solr/analysis/StempelTokenFilterFactory >>>> at java.lang.ClassLoader.defineClass1(Native Method) >>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:634) >>>> at >>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >>>> ... >>>> >>>> Question: What is wrong? :) I use "jar (fastjar) 0.98" to create jars, >>>> I googled on that error but with no answer gave me idea what is wrong >>>> in my .java file. >>>> >>>> Please help, as I believe I am close to the end of that subject. >>>> >>>> Cheers, >>>> Jakub Godawa. >>>> >>>> 2010/11/3 Lance Norskog<goks...@gmail.com>: >>>> >>>>> >>>>> Here's the problem: Solr is a little dumb about these Filter classes, >>>>> and so you have to make a Factory object for the Stempel Filter. >>>>> >>>>> There are a lot of other FilterFactory classes. You would have to just >>>>> copy one and change the names to Stempel and it might actually work. >>>>> >>>>> This will take some Solr programming- perhaps the author can help you? >>>>> >>>>> On Tue, Nov 2, 2010 at 7:08 AM, Jakub Godawa<jakub.god...@gmail.com> >>>>> wrote: >>>>> >>>>>> >>>>>> Sorry, I am not Java programmer at all. I would appreciate more >>>>>> verbose (or step by step) help. >>>>>> >>>>>> 2010/11/2 Bernd Fehling<bernd.fehl...@uni-bielefeld.de>: >>>>>> >>>>>>> >>>>>>> So you call org.getopt.solr.analysis.StempelTokenFilterFactory. >>>>>>> In this case I would assume a file StempelTokenFilterFactory.class >>>>>>> in your directory org/getopt/solr/analysis/. >>>>>>> >>>>>>> And a class which extends the BaseTokenFilterFactory rigth? >>>>>>> ... >>>>>>> public class StempelTokenFilterFactory extends BaseTokenFilterFactory >>>>>>> implements ResourceLoaderAware { >>>>>>> ... >>>>>>> >>>>>>> >>>>>>> >>>>>>> Am 02.11.2010 14:20, schrieb Jakub Godawa: >>>>>>> >>>>>>>> >>>>>>>> This is what stempel-1.0.jar consist of after jar -xf: >>>>>>>> >>>>>>>> jgod...@ubuntu:~/apache-solr-1.4.1/ifaq/lib$ ls -R org/ >>>>>>>> org/: >>>>>>>> egothor getopt >>>>>>>> >>>>>>>> org/egothor: >>>>>>>> stemmer >>>>>>>> >>>>>>>> org/egothor/stemmer: >>>>>>>> Cell.class Diff.class Gener.class MultiTrie2.class >>>>>>>> Optimizer2.class Reduce.class Row.class TestAll.class >>>>>>>> TestLoad.class Trie$StrEnum.class >>>>>>>> Compile.class DiffIt.class Lift.class MultiTrie.class >>>>>>>> Optimizer.class Reduce$Remap.class Stock.class Test.class >>>>>>>> Trie.class >>>>>>>> >>>>>>>> org/getopt: >>>>>>>> stempel >>>>>>>> >>>>>>>> org/getopt/stempel: >>>>>>>> Benchmark.class lucene Stemmer.class >>>>>>>> >>>>>>>> org/getopt/stempel/lucene: >>>>>>>> StempelAnalyzer.class StempelFilter.class >>>>>>>> jgod...@ubuntu:~/apache-solr-1.4.1/ifaq/lib$ ls -R META-INF/ >>>>>>>> META-INF/: >>>>>>>> MANIFEST.MF >>>>>>>> jgod...@ubuntu:~/apache-solr-1.4.1/ifaq/lib$ ls -R res >>>>>>>> res: >>>>>>>> tables >>>>>>>> >>>>>>>> res/tables: >>>>>>>> readme.txt stemmer_1000.out stemmer_100.out stemmer_2000.out >>>>>>>> stemmer_200.out stemmer_500.out stemmer_700.out >>>>>>>> >>>>>>>> 2010/11/2 Bernd Fehling<bernd.fehl...@uni-bielefeld.de>: >>>>>>>> >>>>>>>>> >>>>>>>>> Hi Jakub, >>>>>>>>> >>>>>>>>> if you unzip your stempel-1.0.jar do you have the >>>>>>>>> required directory structure and file in there? >>>>>>>>> org/getopt/stempel/lucene/StempelFilter.class >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Bernd >>>>>>>>> >>>>>>>>> Am 02.11.2010 13:54, schrieb Jakub Godawa: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Erick I've put the jar files like that before. I also added the >>>>>>>>>> directive and put the file in instanceDir/lib >>>>>>>>>> >>>>>>>>>> What is still a problem is that even the files are loaded: >>>>>>>>>> 2010-11-02 13:20:48 org.apache.solr.core.SolrResourceLoader >>>>>>>>>> replaceClassLoader >>>>>>>>>> INFO: Adding >>>>>>>>>> 'file:/home/jgodawa/apache-solr-1.4.1/ifaq/lib/stempel-1.0.jar' >>>>>>>>>> to classloader >>>>>>>>>> >>>>>>>>>> I am not able to use the FilterFactory... maybe I am attempting it >>>>>>>>>> in >>>>>>>>>> a wrong way? >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> Jakub Godawa. >>>>>>>>>> >>>>>>>>>> 2010/11/2 Erick Erickson<erickerick...@gmail.com>: >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> The polish stemmer jar file needs to be findable by Solr, if you >>>>>>>>>>> copy >>>>>>>>>>> it to<solr_home>/lib and restart solr you should be set. >>>>>>>>>>> >>>>>>>>>>> Alternatively, you can add another<lib> directive to the >>>>>>>>>>> solrconfig.xml >>>>>>>>>>> file >>>>>>>>>>> (there are several examples in that file already). >>>>>>>>>>> >>>>>>>>>>> I'm a little confused about not being able to find TokenFilter, >>>>>>>>>>> is that >>>>>>>>>>> still >>>>>>>>>>> a problem? >>>>>>>>>>> >>>>>>>>>>> HTH >>>>>>>>>>> Erick >>>>>>>>>>> >>>>>>>>>>> On Tue, Nov 2, 2010 at 8:07 AM, Jakub >>>>>>>>>>> Godawa<jakub.god...@gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thank you Bernd! I couldn't make it run though. Here is my >>>>>>>>>>>> problem: >>>>>>>>>>>> >>>>>>>>>>>> 1. There is a file ~/apache-solr-1.4.1/ifaq/lib/stempel-1.0.jar >>>>>>>>>>>> 2. In ~/apache-solr-1.4.1/ifaq/solr/conf/solrconfig.xml there is >>>>>>>>>>>> a >>>>>>>>>>>> directive:<lib path="../lib/stempel-1.0.jar" /> >>>>>>>>>>>> 3. In ~/apache-solr-1.4.1/ifaq/solr/conf/schema.xml there is >>>>>>>>>>>> fieldType: >>>>>>>>>>>> >>>>>>>>>>>> (...) >>>>>>>>>>>> <!-- Polish --> >>>>>>>>>>>> <fieldType name="text_pl" class="solr.TextField"> >>>>>>>>>>>> <analyzer> >>>>>>>>>>>> <tokenizer class="solr.WhitespaceTokenizerFactory"/> >>>>>>>>>>>> <filter class="solr.LowerCaseFilterFactory"/> >>>>>>>>>>>> <filter class="org.getopt.stempel.lucene.StempelFilter" /> >>>>>>>>>>>> <!--<filter >>>>>>>>>>>> class="org.getopt.solr.analysis.StempelTokenFilterFactory" >>>>>>>>>>>> protected="protwords.txt" /> --> >>>>>>>>>>>> </analyzer> >>>>>>>>>>>> </fieldType> >>>>>>>>>>>> (...) >>>>>>>>>>>> >>>>>>>>>>>> 4. jar file is loaded but I got an error: >>>>>>>>>>>> SEVERE: Could not start SOLR. Check solr/home property >>>>>>>>>>>> java.lang.NoClassDefFoundError: >>>>>>>>>>>> org/apache/lucene/analysis/TokenFilter >>>>>>>>>>>> at java.lang.ClassLoader.defineClass1(Native Method) >>>>>>>>>>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:634) >>>>>>>>>>>> at >>>>>>>>>>>> >>>>>>>>>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >>>>>>>>>>>> (...) >>>>>>>>>>>> >>>>>>>>>>>> 5. Different class gave me that one: >>>>>>>>>>>> SEVERE: org.apache.solr.common.SolrException: Error loading >>>>>>>>>>>> class >>>>>>>>>>>> 'org.getopt.solr.analysis.StempelTokenFilterFactory' >>>>>>>>>>>> at >>>>>>>>>>>> >>>>>>>>>>>> org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:375) >>>>>>>>>>>> at >>>>>>>>>>>> >>>>>>>>>>>> org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:390) >>>>>>>>>>>> (...) >>>>>>>>>>>> >>>>>>>>>>>> Question is: How to make<fieldType /> and<filter /> work with >>>>>>>>>>>> that >>>>>>>>>>>> Stempel? :) >>>>>>>>>>>> >>>>>>>>>>>> Cheers, >>>>>>>>>>>> Jakub Godawa. >>>>>>>>>>>> >>>>>>>>>>>> 2010/10/29 Bernd Fehling<bernd.fehl...@uni-bielefeld.de>: >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Hi Jakub, >>>>>>>>>>>>> >>>>>>>>>>>>> I have ported the KStemmer for use in most recent Solr trunk >>>>>>>>>>>>> version. >>>>>>>>>>>>> My stemmer is located in the lib directory of Solr >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> "solr/lib/KStemmer-2.00.jar" >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> because it belongs to Solr. >>>>>>>>>>>>> >>>>>>>>>>>>> Write it as FilterFactory and use it as Filter like: >>>>>>>>>>>>> <filter class="de.ubbielefeld.solr.analysis.KStemFilterFactory" >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> protected="protwords.txt" /> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> This is how my fieldType looks like: >>>>>>>>>>>>> >>>>>>>>>>>>> <fieldType name="text_kstem" class="solr.TextField" >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> positionIncrementGap="100"> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> <analyzer type="index"> >>>>>>>>>>>>> <tokenizer class="solr.WhitespaceTokenizerFactory" /> >>>>>>>>>>>>> <filter class="solr.StopFilterFactory" ignoreCase="true" >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> words="stopwords.txt" enablePositionIncrements="false" /> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> <filter class="solr.WordDelimiterFilterFactory" >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> generateWordParts="1" generateNumberParts="1" catenateWords="1" >>>>>>>>>>>> catenateNumbers="1" >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> catenateAll="0" splitOnCaseChange="1" /> >>>>>>>>>>>>> <filter class="solr.LowerCaseFilterFactory" /> >>>>>>>>>>>>> <filter >>>>>>>>>>>>> class="de.ubbielefeld.solr.analysis.KStemFilterFactory" >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> protected="protwords.txt" /> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> <filter class="solr.RemoveDuplicatesTokenFilterFactory" >>>>>>>>>>>>> /> >>>>>>>>>>>>> </analyzer> >>>>>>>>>>>>> <analyzer type="query"> >>>>>>>>>>>>> <tokenizer class="solr.WhitespaceTokenizerFactory" /> >>>>>>>>>>>>> <filter class="solr.StopFilterFactory" ignoreCase="true" >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> words="stopwords.txt" /> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> <filter class="solr.WordDelimiterFilterFactory" >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> generateWordParts="1" generateNumberParts="1" catenateWords="0" >>>>>>>>>>>> catenateNumbers="0" >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> catenateAll="0" splitOnCaseChange="1" /> >>>>>>>>>>>>> <filter class="solr.LowerCaseFilterFactory" /> >>>>>>>>>>>>> <filter >>>>>>>>>>>>> class="de.ubbielefeld.solr.analysis.KStemFilterFactory" >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> protected="protwords.txt" /> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> <filter class="solr.RemoveDuplicatesTokenFilterFactory" >>>>>>>>>>>>> /> >>>>>>>>>>>>> </analyzer> >>>>>>>>>>>>> </fieldType> >>>>>>>>>>>>> >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> Bernd >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Am 28.10.2010 14:56, schrieb Jakub Godawa: >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi! >>>>>>>>>>>>>> There is a polish stemmer http://www.getopt.org/stempel/ and I >>>>>>>>>>>>>> have >>>>>>>>>>>>>> problems connecting it with solr 1.4.1 >>>>>>>>>>>>>> Questions: >>>>>>>>>>>>>> >>>>>>>>>>>>>> 1. Where EXACTLY do I put "stemper-1.0.jar" file? >>>>>>>>>>>>>> 2. How do I register the file, so I can build a fieldType >>>>>>>>>>>>>> like: >>>>>>>>>>>>>> >>>>>>>>>>>>>> <fieldType name="text_pl" class="solr.TextField"> >>>>>>>>>>>>>> <analyzer >>>>>>>>>>>>>> class="org.geoopt.solr.analysis.StempelTokenFilterFactory"/> >>>>>>>>>>>>>> </fieldType> >>>>>>>>>>>>>> >>>>>>>>>>>>>> 3. Is that the right approach to make it work? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for verbose explanation, >>>>>>>>>>>>>> Jakub. >>>>>>>>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Lance Norskog >>>>> goks...@gmail.com >>>>> >>>>> >>>> >>>> >>> >>> >>> -- >>> Lance Norskog >>> goks...@gmail.com >>> >>> >