Encoding issue with solr querying
I have indexed some non-english pages in solr index. From the solr admin
interface I can query using the raw unicodes like this,
\u062d\u06c2 \u0627\u0648\u0644\u202c
and this gives the resulting page correctly. Now I'm trying to do that
through java script in a web page via SolrJ for english queries it is
working fine but not for non-english content.
If I've to use the above unicoded query then how shall I proceed. I've a
method to which I pass the query which returns back the result after quering
solr. I even tried encoding the query in UTF-8 but of no use. The web page
is like this
*
*
setQuery() is part of solrJave class that just sets the query string and
getQuery() function is emebeded inside the javascript like this,
*
function getQuery() {
return (encodeUTF8("non-english-query")); //my query, how to
put the unicoded query here? for any english queries its working fine.
}
*
I tried using some encode/decode for utf-8, the functions are given below,
*
function encodeUTF8( s ){
return unescape( encodeURIComponent( s ) );
}
function decodeUTF8( s ){
return decodeURIComponent( escape( s ) );
}
*
Any advice will be highly appreciated.
KK
Re: unable to run the solr in tomcat 5.0
It would be easier for anyone to give you a precise answer if you can be more specific about the steps you followed for installing solr/tomcat and where exactly you got stuck. KK On Wed, May 6, 2009 at 11:58 AM, uday kumar maddigatla wrote: > > Hi, > > I'm new to this Solr. I got distribution of Solr. i placed the war file in > tomcat/webapps. > > After that i don't know what to do. I got confused while reading The > instalation notes which is given in wiki . > > What is the next step to run the Solr. > > I really got confused with the lines in wiki. > > Please help me with instalation instructions. > -- > View this message in context: > http://www.nabble.com/unable-to-run-the-solr-in-tomcat-5.0-tp23400759p23400759.html > Sent from the Solr - User mailing list archive at Nabble.com. > >
Re: Encoding issue with solr querying
Can someone tell me how to query for some raw unicode text[the one mentioned
in my last mail] through SolrJ?
Thanks,
KK.
On Tue, May 5, 2009 at 6:40 PM, KK wrote:
> I have indexed some non-english pages in solr index. From the solr admin
> interface I can query using the raw unicodes like this,
>
> \u062d\u06c2 \u0627\u0648\u0644\u202c
>
> and this gives the resulting page correctly. Now I'm trying to do that
> through java script in a web page via SolrJ for english queries it is
> working fine but not for non-english content.
> If I've to use the above unicoded query then how shall I proceed. I've a
> method to which I pass the query which returns back the result after quering
> solr. I even tried encoding the query in UTF-8 but of no use. The web page
> is like this
>
> *
>
> onClick="document.solrJava.setQuery(getQuery());">
>
> MAYSCRIPT id="talk">
>
>
>
> *
> setQuery() is part of solrJave class that just sets the query string and
> getQuery() function is emebeded inside the javascript like this,
>
> *
> function getQuery() {
> return (encodeUTF8("non-english-query")); //my query, how to
> put the unicoded query here? for any english queries its working fine.
> }
> *
>
> I tried using some encode/decode for utf-8, the functions are given below,
>
> *
> function encodeUTF8( s ){
> return unescape( encodeURIComponent( s ) );
> }
> function decodeUTF8( s ){
> return decodeURIComponent( escape( s ) );
> }
> *
>
>
> Any advice will be highly appreciated.
>
> KK
>
What are the Unicode encodings supported by Solr?
Hi, I'd like to know about the different Unicode[/any other?] encodings supported by Solr for posting docs [thru Solrj in my case]. Is it that just UTF-8, UCN supported or other character encodings like NCR(decimal), NCR(hex) etc are supported as well? Now the problem is that during automating the crawling and indexing process for Solr I found that for most of the pages the encoding is UTF-8[in this case searching works fine] but for others the encoding is some other character encoding[like NCR(dec), NCR(hex) or might be something else, don't have much idea on this]. So when I fetch the page content thru java methods using InputSteamReaders and after stripping various tags what I obtained is raw text with some encoding not getting supported by Solr. So either I've to confing Solr to support these other encodings as well[only if it is possible] otherwise convert whatever is the raw text to UTF-8 using some standard encoders[this solution seems better to me, provided I'm able to detect the encoding for input]. I'd like to know if there are some standard encoders are available for this purpose[must be right? din't google much]. Any advice on this is highly appreciated. An off-beat Q: In some of the pages I'm getting some \ufffd chars which I think is some sort of unmappable[by Java?] character, right?. Any idea on how to handle this? Just replacing with blank char will not do [this depends on the requirement, though]. Thanks, KK.
Restarting tomcat deletes all Solr indexes
Hi, I'm facing a silly problem. Every time I restart tomcat all the indexes are lost. I used all the default configurations. I'm pretty sure there must be some basic changes to fix this. I'd highly appreciate if someone could direct me fixing this. Thanks, KK.
Re: Restarting tomcat deletes all Solr indexes
Thanks for your response @aklochkov. But I again noticed that something is wrong in my solr/tomcat config[I spent a lot of time making solr run], b'coz in the solr admin page [ http://localhost:8080/solr/admin/] what I see is that the $CWD is the location where from I restarted tomcat and seems this $cwd gets picked and used for index data[Is it the default behavior? or something wrong from my side?, or may be I'm asking some stupid question ]. Once I was in /etc and from there I restarted the tomcat and when I tried to open the solr admin page I found an error saying that can not create index directory some permission issue I think [it gave a directory str like /etc/solr/index ... ]. I'm pretty sure something is wrong in configuration. One more thing assures me about this is the fact that I found many solr index directories here and there[ these are I think the locations where I was when I restarted tomcat at that time ]. Earlier I was using the java_opts to set the solr home like this export JAVA_OPTS="$JAVA_OPTS -D/usr/local/solr"#in .bashrc but I commented that and instead added the jndi entry in /usr/local/tomcat/webapps/solr/WEB-INF/web.xml as this solr/home /usr/local/solr java.lang.String Even the entry SolrHome in solr admin page say that SolrHome is "/usr/loca/solr" but the index gets created in $CWD. Is it the case that I created entries for SolrHome in multiple places? which is obviously wrong. Can someone point me what is the issue. Thank you very much. --KK On Tue, May 12, 2009 at 2:39 PM, Andrey Klochkov wrote: > Hi, > > I know that when starting Solr checks index directory existence, and > creates > new fresh index if it doesn't exist. Does it help? If no, the next step I'd > do in your case is patching SolrCore.initIndex method - insert some > logging, > or run EmbeddedSolrServer with debugger etc. > > On Mon, May 11, 2009 at 1:25 PM, KK wrote: > > > Hi, > > I'm facing a silly problem. Every time I restart tomcat all the indexes > are > > lost. I used all the default configurations. I'm pretty sure there must > be > > some basic changes to fix this. I'd highly appreciate if someone could > > direct me fixing this. > > > > Thanks, > > KK. > > > > > -- > Andrew Klochkov >
Re: Restarting tomcat deletes all Solr indexes
One more information I would like to add. The entry in solr stats page says this: readerDir : org.apache.lucene.store.FSDirectory@/home/kk/solr/data/index when I ran from /home/kk and this: readerDir : org.apache.lucene.store.FSDirectory@ /home/kk/junk/solr/data/index after running from /home/kk/junk That assures the me the problem, but what is the solution? Thanks, KK. On Tue, May 12, 2009 at 4:41 PM, KK wrote: > Thanks for your response @aklochkov. > But I again noticed that something is wrong in my solr/tomcat config[I > spent a lot of time making solr run], b'coz in the solr admin page [ > http://localhost:8080/solr/admin/] what I see is that the $CWD is the > location where from I restarted tomcat and seems this $cwd gets picked and > used for index data[Is it the default behavior? or something wrong from my > side?, or may be I'm asking some stupid question ]. > Once I was in /etc and from there I restarted the tomcat and when I tried > to open the solr admin page I found an error saying that can not create > index directory some permission issue I think [it gave a directory str like > /etc/solr/index ... ]. I'm pretty sure something is wrong in configuration. > One more thing assures me about this is the fact that I found many solr > index directories here and there[ these are I think the locations where I > was when I restarted tomcat at that time ]. Earlier I was using the > java_opts to set the solr home like this > > export JAVA_OPTS="$JAVA_OPTS -D/usr/local/solr"#in .bashrc > > but I commented that and instead added the jndi entry in > /usr/local/tomcat/webapps/solr/WEB-INF/web.xml as this > > >solr/home >/usr/local/solr >java.lang.String > > > Even the entry SolrHome in solr admin page say that SolrHome is > "/usr/loca/solr" but the index gets created in $CWD. Is it the case that I > created entries for SolrHome in multiple places? which is obviously wrong. > Can someone point me what is the issue. Thank you very much. > > --KK > > > > On Tue, May 12, 2009 at 2:39 PM, Andrey Klochkov < > akloch...@griddynamics.com> wrote: > >> Hi, >> >> I know that when starting Solr checks index directory existence, and >> creates >> new fresh index if it doesn't exist. Does it help? If no, the next step >> I'd >> do in your case is patching SolrCore.initIndex method - insert some >> logging, >> or run EmbeddedSolrServer with debugger etc. >> >> On Mon, May 11, 2009 at 1:25 PM, KK wrote: >> >> > Hi, >> > I'm facing a silly problem. Every time I restart tomcat all the indexes >> are >> > lost. I used all the default configurations. I'm pretty sure there must >> be >> > some basic changes to fix this. I'd highly appreciate if someone could >> > direct me fixing this. >> > >> > Thanks, >> > KK. >> > >> >> >> -- >> Andrew Klochkov >> > >
Max no of solr cores supported and how to restrict a query to a particular core?
I want to know the maximum no of cores supported by Solr. 1000s or may be millions all under one solr instance ? Also I want to know how to redirect a particular query to a particular core. Actually I'm querying solr from Ajax, so I think there must be some request parameter that says which core we want to query, right? Can some one tell me how to do this, any good pointers on the same will be helpful as well. Thank you. --kk
Re: Max no of solr cores supported and how to restrict a query to a particular core?
Thank you very much. Got the point. One off the track question, can we automate the creation of new cores[it requires manually editing the solr.xml file as I know, and what about the location of core index directory, do we need to point that manually as well]. After going through the wiki what I found is we've to mention the names of cores in solr.xml. I want to automate the process in such a way that when a user registers[ on say my site for the service], we'll create a coresponding core for the same user and with a specific core id[unique for this user only] so that the user will be given a search interface that will redirect all searches for this user to http://host:port//select Will apprecite any ideas on this. Thanks, KK. 2009/5/14 Noble Paul നോബിള് नोब्ळ् > there is no hard limit on the no:of cores. it is limited by your > system's ability to open files and the resources. > the queries are automatically sent to appropriate core if your url is > > htt://host:port//select > > On Thu, May 14, 2009 at 1:58 PM, KK wrote: > > I want to know the maximum no of cores supported by Solr. 1000s or may be > > millions all under one solr instance ? > > Also I want to know how to redirect a particular query to a particular > core. > > Actually I'm querying solr from Ajax, so I think there must be some > request > > parameter that says which core we want to query, right? Can some one tell > me > > how to do this, any good pointers on the same will be helpful as well. > > Thank you. > > > > --kk > > > > > > -- > - > Noble Paul | Principal Engineer| AOL | http://aol.com >
Re: Max no of solr cores supported and how to restrict a query to a particular core?
Thank you very much. LOL, Its in the same wiki I was told to go through. I've a question regarding creating ofsolr cores on the fly. The wiki says, .Creates a new core and register it. If persistence is enabled (persist=true), the configuration for this new core will be saved in 'solr.xml'. If a core with the same name exists, while the "new" created core is initializing, the "old" one will continue to accept requests. Once it has finished, all new request will go to the "new" core, and the "old" core will be unloaded. So I've to wait for some time [say a couple of secs, may be less than that] before I start adding pages to that core. I think this is the way to handle it , otherwise some content which should have been indexed by the new core, will get indexed by the existing core[as the wiki says], which I don't want to happen. Any other ideas for handling the same. Thanks, KK. 2009/5/14 Noble Paul നോബിള് नोब्ळ् > Solr already supports this . > please refer this > > http://wiki.apache.org/solr/CoreAdmin#head-7ca1b98a9df8b8ca0dcfbfc49940ed5ac98c4a08 > > ensure that your solr.xml is persistent > > http://wiki.apache.org/solr/CoreAdmin#head-7508c24c6e2dadad2dfea39b2fba045062481da8 > > On Thu, May 14, 2009 at 3:43 PM, KK wrote: > > Thank you very much. Got the point. > > One off the track question, can we automate the creation of new cores[it > > requires manually editing the solr.xml file as I know, and what about the > > location of core index directory, do we need to point that manually as > > well]. > > After going through the wiki what I found is we've to mention the names > of > > cores in solr.xml. I want to automate the process in such a way that when > a > > user registers[ on say my site for the service], we'll create a > coresponding > > core for the same user and with a specific core id[unique for this user > > only] so that the user will be given a search interface that will > redirect > > all searches for this user to http://host:port/ this > > user>/select > > Will apprecite any ideas on this. > > > > Thanks, > > KK. > > > > 2009/5/14 Noble Paul നോബിള് नोब्ळ् > > > >> there is no hard limit on the no:of cores. it is limited by your > >> system's ability to open files and the resources. > >> the queries are automatically sent to appropriate core if your url is > >> > >> htt://host:port//select > >> > >> On Thu, May 14, 2009 at 1:58 PM, KK wrote: > >> > I want to know the maximum no of cores supported by Solr. 1000s or may > be > >> > millions all under one solr instance ? > >> > Also I want to know how to redirect a particular query to a particular > >> core. > >> > Actually I'm querying solr from Ajax, so I think there must be some > >> request > >> > parameter that says which core we want to query, right? Can some one > tell > >> me > >> > how to do this, any good pointers on the same will be helpful as well. > >> > Thank you. > >> > > >> > --kk > >> > > >> > >> > >> > >> -- > >> - > >> Noble Paul | Principal Engineer| AOL | http://aol.com > >> > > > > > > -- > - > Noble Paul | Principal Engineer| AOL | http://aol.com >
Sole core naming convention for multicores
Hi All, I'm trying to put multicores for Solr[lol, finding the multicore config a bit difficult, any good/simple steps to do the same?any pointers]. Let me come to the point, essentially what I want is that whenever a person registersfor our service, I'll use his mail-id[this is unique] as the corename. I dont know if its viable or not. As per the wiki example the creation/registration of new core is done like this, http://localhost:8983/solr/admin/cores?action=CREATE&name=coreX&instanceDir=path_to_instance_directory&config=config_file_name.xml&schema=schem_file_name.xml&dataDir=data this says the name as something like coreX where X replaces a num. Is it possible to have a name like say "alex...@abc.com"? If not may be I've map the mail-id to some unique number that I'll use as a core name. I don't want to do all this [don't know either], hence my question. Do let me know some smart ways of doing the same. Note: I've to use mail-id as the unique identifier. Thanks in appreciation. Thanks, KK>
Re: Sole core naming convention for multicores
Thank you Otis. One silly question, how would I know that a particular character is forbidden, I think Solr will give me exceptions saying that some characters not allowed, right? Thank, KK. On Sun, May 17, 2009 at 3:12 AM, Otis Gospodnetic < otis_gospodne...@yahoo.com> wrote: > > KK, > > That should work just fine. Should any of the characters in email > addresses turn out to be forbidden, just replace them consistently. For > example, if @ turns out to be the problem, you could simple replace it with > _. > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > - Original Message > > From: KK > > To: solr-user@lucene.apache.org > > Sent: Saturday, May 16, 2009 3:45:01 AM > > Subject: Sole core naming convention for multicores > > > > Hi All, > > I'm trying to put multicores for Solr[lol, finding the multicore config a > > bit difficult, any good/simple steps to do the same?any pointers]. > > Let me come to the point, essentially what I want is that whenever a > person > > registersfor our service, I'll use his mail-id[this is unique] as the > > corename. I dont know if its viable or not. As per the wiki example the > > creation/registration of new core is done like this, > > > > > http://localhost:8983/solr/admin/cores?action=CREATE&name=coreX&instanceDir=path_to_instance_directory&config=config_file_name.xml&schema=schem_file_name.xml&dataDir=data > > > > this says the name as something like coreX where X replaces a num. Is it > > possible to have a name like say "alex...@abc.com"? If not may be I've > map > > the mail-id to some unique number that I'll use as a core name. I don't > want > > to do all this [don't know either], hence my question. Do let me know > some > > smart ways of doing the same. > > Note: I've to use mail-id as the unique identifier. Thanks in > appreciation. > > > > Thanks, > > KK> > >
Simplest way of registering new solr core!
Hi, What is the simplest way of registering a new solr core? Do we have to use some standard APIs for this, or making a Ajax get call to http://localhost:8983/solr/admin/cores with proper request parameter lik these, [?action=CREATE&name=coreX&instanceDir=path_to_instance_directory&config=config_file_name.xml&schema=schem_file_name.xml&dataDir=data] will do the job. I think this should work. Correct me if I'm wrong The process remains same for other requests as well[ like getting core status, reload, rename etc], right? Also, I would be thankful if someone can point me to some good tutorial on adding new cores to the existing ones, configuring multicore indexing etc. I don't find the solr wiki that useful for doing the same. Thanks, KK 2009/5/14 Noble Paul നോബിള് नोब्ळ् > usually it happens fast . and the request returns only after the cor > creation is completed. So if you are doing all the operations in the > same thread no need to wait. > > > > > On Thu, May 14, 2009 at 7:37 PM, KK wrote: > > > > Thank you very much. LOL, Its in the same wiki I was told to go through. > > I've a question regarding creating ofsolr cores on the fly. The wiki > says, > > > > .Creates a new core and register it. If persistence is enabled > > (persist=true), the configuration for this new core will be saved in > > 'solr.xml'. If a core with the same name exists, while the "new" created > > core is initializing, the "old" one will continue to accept requests. > Once > > it has finished, all new request will go to the "new" core, and the "old" > > core will be unloaded. > > > > So I've to wait for some time [say a couple of secs, may be less than > that] > > before I start adding pages to that core. I think this is the way to > handle > > it , otherwise some content which should have been indexed by the new > core, > > will get indexed by the existing core[as the wiki says], which I don't > want > > to happen. Any other ideas for handling the same. > > > > > > Thanks, > > KK. > > > > 2009/5/14 Noble Paul നോബിള് नोब्ळ् > >> > >> Solr already supports this . > >> please refer this > >> > >> > http://wiki.apache.org/solr/CoreAdmin#head-7ca1b98a9df8b8ca0dcfbfc49940ed5ac98c4a08 > >> > >> ensure that your solr.xml is persistent > >> > >> > http://wiki.apache.org/solr/CoreAdmin#head-7508c24c6e2dadad2dfea39b2fba045062481da8 > >> > >> On Thu, May 14, 2009 at 3:43 PM, KK wrote: > >> > Thank you very much. Got the point. > >> > One off the track question, can we automate the creation of new > cores[it > >> > requires manually editing the solr.xml file as I know, and what about > >> > the > >> > location of core index directory, do we need to point that manually as > >> > well]. > >> > After going through the wiki what I found is we've to mention the > names > >> > of > >> > cores in solr.xml. I want to automate the process in such a way that > >> > when a > >> > user registers[ on say my site for the service], we'll create a > >> > coresponding > >> > core for the same user and with a specific core id[unique for this > user > >> > only] so that the user will be given a search interface that will > >> > redirect > >> > all searches for this user to http://host:port/ >> > this > >> > user>/select > >> > Will apprecite any ideas on this. > >> > > >> > Thanks, > >> > KK. > >> > > >> > 2009/5/14 Noble Paul നോബിള് नोब्ळ् > >> > > >> >> there is no hard limit on the no:of cores. it is limited by your > >> >> system's ability to open files and the resources. > >> >> the queries are automatically sent to appropriate core if your url is > >> >> > >> >> htt://host:port//select > >> >> > >> >> On Thu, May 14, 2009 at 1:58 PM, KK > wrote: > >> >> > I want to know the maximum no of cores supported by Solr. 1000s or > >> >> > may be > >> >> > millions all under one solr instance ? > >> >> > Also I want to know how to redirect a particular query to a > >> >> > particular > >> >> core. > >> >> > Actually I'm querying solr from Ajax, so I think there must be some > >> >> request > >> >> > parameter that says which core we want to query, right? Can some > one > >> >> > tell > >> >> me > >> >> > how to do this, any good pointers on the same will be helpful as > >> >> > well. > >> >> > Thank you. > >> >> > > >> >> > --kk > >> >> > > >> >> > >> >> > >> >> > >> >> -- > >> >> - > >> >> Noble Paul | Principal Engineer| AOL | http://aol.com > >> >> > >> > > >> > >> > >> > >> -- > >> - > >> Noble Paul | Principal Engineer| AOL | http://aol.com > > > > > > > > -- > - > Noble Paul | Principal Engineer| AOL | http://aol.com >
Re: Simplest way of registering new solr core!
Do we have to extend the CoreAdminHandler class and use some methods therein to register a new core? Thanks in appreciation. --KK On Mon, May 18, 2009 at 12:48 PM, KK wrote: > Hi, > What is the simplest way of registering a new solr core? Do we have to use > some standard APIs for this, or making a Ajax get call to > http://localhost:8983/solr/admin/cores > > with proper request parameter lik these, > > [?action=CREATE&name=coreX&instanceDir=path_to_instance_directory&config=config_file_name.xml&schema=schem_file_name.xml&dataDir=data] > will do the job. I think this should work. Correct me if I'm wrong > The process remains same for other requests as well[ like getting core > status, reload, rename etc], right? > > Also, I would be thankful if someone can point me to some good tutorial on > adding new cores to the existing ones, configuring multicore indexing etc. I > don't find the solr wiki that useful for doing the same. > > > Thanks, > KK > > > 2009/5/14 Noble Paul നോബിള് नोब्ळ् > >> usually it happens fast . and the request returns only after the cor >> creation is completed. So if you are doing all the operations in the >> same thread no need to wait. >> >> >> >> >> On Thu, May 14, 2009 at 7:37 PM, KK wrote: >> > >> > Thank you very much. LOL, Its in the same wiki I was told to go through. >> > I've a question regarding creating ofsolr cores on the fly. The wiki >> says, >> > >> > .Creates a new core and register it. If persistence is enabled >> > (persist=true), the configuration for this new core will be saved in >> > 'solr.xml'. If a core with the same name exists, while the "new" created >> > core is initializing, the "old" one will continue to accept requests. >> Once >> > it has finished, all new request will go to the "new" core, and the >> "old" >> > core will be unloaded. >> > >> > So I've to wait for some time [say a couple of secs, may be less than >> that] >> > before I start adding pages to that core. I think this is the way to >> handle >> > it , otherwise some content which should have been indexed by the new >> core, >> > will get indexed by the existing core[as the wiki says], which I don't >> want >> > to happen. Any other ideas for handling the same. >> > >> > >> > Thanks, >> > KK. >> > >> > 2009/5/14 Noble Paul നോബിള് नोब्ळ् >> >> >> >> Solr already supports this . >> >> please refer this >> >> >> >> >> http://wiki.apache.org/solr/CoreAdmin#head-7ca1b98a9df8b8ca0dcfbfc49940ed5ac98c4a08 >> >> >> >> ensure that your solr.xml is persistent >> >> >> >> >> http://wiki.apache.org/solr/CoreAdmin#head-7508c24c6e2dadad2dfea39b2fba045062481da8 >> >> >> >> On Thu, May 14, 2009 at 3:43 PM, KK >> wrote: >> >> > Thank you very much. Got the point. >> >> > One off the track question, can we automate the creation of new >> cores[it >> >> > requires manually editing the solr.xml file as I know, and what about >> >> > the >> >> > location of core index directory, do we need to point that manually >> as >> >> > well]. >> >> > After going through the wiki what I found is we've to mention the >> names >> >> > of >> >> > cores in solr.xml. I want to automate the process in such a way that >> >> > when a >> >> > user registers[ on say my site for the service], we'll create a >> >> > coresponding >> >> > core for the same user and with a specific core id[unique for this >> user >> >> > only] so that the user will be given a search interface that will >> >> > redirect >> >> > all searches for this user to http://host:port/> >> > this >> >> > user>/select >> >> > Will apprecite any ideas on this. >> >> > >> >> > Thanks, >> >> > KK. >> >> > >> >> > 2009/5/14 Noble Paul നോബിള് नोब्ळ् >> >> > >> >> >> there is no hard limit on the no:of cores. it is limited by your >> >> >> system's ability to open files and the resources. >> >> >> the queries are automatically sent to appropriate core
Re: Sole core naming convention for multicores
I think I should also do the same way. Thanks Brain for pointing me to this idea. As per my db design I don't have any single key as PK, I'm thinking of putting a new field called [say] coreId and make it PK with auto-increment option[I'm using MySQL, btw], and this will solve the problem. I think this is simple yet elegant way to fix the problem. Thanks, KK On Mon, May 18, 2009 at 8:14 PM, Brian Mansell wrote: > KK - > > In my experience with multi-core, I've found that using the user record's > integer PK for each user core works well by still allowing the user to > update their email addresses / usernames over time. > > cheers, > > --bemansell > > > On May 17, 2009 10:39 PM, "KK" wrote: > > Thank you Otis. > One silly question, how would I know that a particular character is > forbidden, I think Solr will give me exceptions saying that some characters > not allowed, right? > > Thank, > KK. > > On Sun, May 17, 2009 at 3:12 AM, Otis Gospodnetic < > otis_gospodne...@yahoo.com> wrote: > > KK, > >... >
Re: Simplest way of registering new solr core!
Thanks. I'm still finding it difficult to move from unicore to multicore. What I understood so far is that we've to put a solr.xml file in $solr.home and this is the config file loaded first, if this is not available then solr looks for solrconfig.xml under coreX directory withing multicore directory. Am I right? Earlier I was using a file name solr@tomcat.home/conf/catalina/localhost to specify $solr.home directory. If I remove that and put solr.xml under $solr.home and start tomcat accessing to solr from borwser gives me error saying path not set for core. Can someone help me fixing this. Do I need to remove the solr.xml file from @Tomcat.home/conf/catalina/localhost/ and keep the solr.xml under $solr.home. As per my requirement I want to be able to register new cores on-the-fly and start posting new docs to the newly registered core simultaneously with other existing cores. Thank you very much. KK. 2009/5/18 Noble Paul നോബിള് नोब्ळ् > On Mon, May 18, 2009 at 4:06 PM, KK wrote: > > Do we have to extend the CoreAdminHandler class and use some methods > therein > > to register a new core? > no. th create command will do that > > Thanks in appreciation. > > > > --KK > > > > On Mon, May 18, 2009 at 12:48 PM, KK wrote: > > > >> Hi, > >> What is the simplest way of registering a new solr core? Do we have to > use > >> some standard APIs for this, or making a Ajax get call to > >> http://localhost:8983/solr/admin/cores > >> > >> with proper request parameter lik these, > >> > >> > [?action=CREATE&name=coreX&instanceDir=path_to_instance_directory&config=config_file_name.xml&schema=schem_file_name.xml&dataDir=data] > >> will do the job. I think this should work. Correct me if I'm wrong > >> The process remains same for other requests as well[ like getting core > >> status, reload, rename etc], right? > >> > >> Also, I would be thankful if someone can point me to some good tutorial > on > >> adding new cores to the existing ones, configuring multicore indexing > etc. I > >> don't find the solr wiki that useful for doing the same. > >> > >> > >> Thanks, > >> KK > >> > >> > >> 2009/5/14 Noble Paul നോബിള് नोब्ळ् > >> > >>> usually it happens fast . and the request returns only after the cor > >>> creation is completed. So if you are doing all the operations in the > >>> same thread no need to wait. > >>> > >>> > >>> > >>> > >>> On Thu, May 14, 2009 at 7:37 PM, KK > wrote: > >>> > > >>> > Thank you very much. LOL, Its in the same wiki I was told to go > through. > >>> > I've a question regarding creating ofsolr cores on the fly. The wiki > >>> says, > >>> > > >>> > .Creates a new core and register it. If persistence is enabled > >>> > (persist=true), the configuration for this new core will be saved in > >>> > 'solr.xml'. If a core with the same name exists, while the "new" > created > >>> > core is initializing, the "old" one will continue to accept requests. > >>> Once > >>> > it has finished, all new request will go to the "new" core, and the > >>> "old" > >>> > core will be unloaded. > >>> > > >>> > So I've to wait for some time [say a couple of secs, may be less than > >>> that] > >>> > before I start adding pages to that core. I think this is the way to > >>> handle > >>> > it , otherwise some content which should have been indexed by the new > >>> core, > >>> > will get indexed by the existing core[as the wiki says], which I > don't > >>> want > >>> > to happen. Any other ideas for handling the same. > >>> > > >>> > > >>> > Thanks, > >>> > KK. > >>> > > >>> > 2009/5/14 Noble Paul നോബിള് नोब्ळ् > >>> >> > >>> >> Solr already supports this . > >>> >> please refer this > >>> >> > >>> >> > >>> > http://wiki.apache.org/solr/CoreAdmin#head-7ca1b98a9df8b8ca0dcfbfc49940ed5ac98c4a08 > >>> >> > >>> >> ensure that your solr.xml is persistent > >>> >> > >>> >> > >>> > http://wiki.apache.org/solr/CoreAd
What are the basic requirements for on-the-fly registration/creation of new Core?
Hi All, Finally I made the multicore thing running by copying the core0 and core1 directory to $solr.home and putting a solr.xml file under the same directory. A bit relieved ! I would like to do on-the-fly registration of core[I've provision to add unique corenames everytime I register a new one] and as per the basic requirements mentioned in the wiki[http://wiki.apache.org/solr/CoreAdmin] I did the following: 1. set persistent="true" and sharedLIb="lib" For this I actually copied the file given in wiki and put it under solr.home. Started tomcat and tried to register a new core named core3 like this[I'm running solr on 8080] http://localhost:8080/solr/admin/cores?action=CREATE&name=core2 and the browser gave me error saying that instanceDir is missing, Agreed its missing but do I need to provide the path to a new directory everytime a register a new core or any old core'x directory will do the job? Then I copied an existing directory core0 to core2 with all its /bin and /conf and other stuffs. Then I tried to refresh the earlier page in firefox and WOW it worked. Then to confirm that the new core details has been added to solr.xml I cross checked the file and found a new entry for core2. Its working. Now let me mention what I want to do, # I want multiple cores, true but all the core will essentially have the same schema and config as well. # I want separete index for each core so that during posting and searching I can search a specific core Now I've certain doubts about these things. 1. Do we have to create a new directory with all those /bin and /conf everytime we've to register a new core. This sounds silly and I'm pretty sure I must be wrong . There must be some way to just say that : Create this new core having the core name as ABXZ. The wiki does say the last three parameters are optinal [Wiki:Note that config ,schema & dataDir parameters are optional.] Can't we have some base schema and config file say under $solr.home [or may be somewhere else if that creates problem] and everytime we create a new core just point to those. Can someone give me direct examples for the same[the wiki does say about this but I'm not completely clear about that, some examples will do the job]. 2. Do we have to make sure that the data directory is sitting inside coreX directory, or we can put it elsewhere? Thank you very much. KK.
Re: Defining DataDir in Multi-Core
I could not follow[is this mail a continuation of some old mail, a part of which seems to be missing], but I want to. Is it the case that CREATE is to be supported by solr1.4 i.e currently solr1.3 doesnot support this? Correct me if I'm wrong . Vivek could you please tell me how did you fix the problem of using a single schema and config file for all cores and having different data directories. I'm stuck at the same point as you were. Please help me out. Can you provide some specific examples that shows the way you used the create statement to register new cores on the fly. Thank you . --KK On Tue, May 19, 2009 at 1:17 PM, vivek sar wrote: > Yeah, it was sometime back - it did work. Thanks for following up. > > On Tue, May 19, 2009 at 12:34 AM, RaghavPrabhu > wrote: > > > > Hi Vivek, > > > > Have you figure out the problem of creating the data dir in wrong > > location? > > > > For me its working... > > > > Just comment the data dir (in solrconfig.xml file) and create the core > > via REST call. It should work!!! > > > > Thanks & regards > > Prabhu.K > > > > > > > > vivek sar wrote: > >> > >> Hi, > >> > >> I tried the latest nightly build (04-01-09) - it takes the dataDir > >> property now, but it's creating the Data dir at the wrong location. > >> For ex., I've the following in solr.xml, > >> > >> > >> > >>>> dataDir="/Users/opal/temp/afterchat/solr/data/core0"/> > >> > >> > >> > >> but, it always seem to be creating the solr/data directory in the cwd > >> (where I started the Tomcat from). Here is the log from Catalina.out, > >> > >> Apr 1, 2009 10:47:21 AM org.apache.solr.core.SolrCore > >> INFO: [core2] Opening new SolrCore at /Users/opal/temp/chat/solr/, > >> dataDir=./solr/data/ > >> .. > >> Apr 1, 2009 10:47:21 AM org.apache.solr.core.SolrCore initIndex > >> WARNING: [core2] Solr index directory './solr/data/index' doesn't > >> exist. Creating new index... > >> > >> I've also tried relative paths, but to no avail. > >> > >> Is this a bug? > >> > >> Thanks, > >> -vivek > >> > >> On Wed, Apr 1, 2009 at 9:45 AM, vivek sar wrote: > >>> Thanks Shalin. > >>> > >>> Is it available in the latest nightly build? > >>> > >>> Is there any other way I can create cores dynamically (using CREATE > >>> service) which will use the same schema.xml and solrconfig.xml, but > >>> write to different data directories? > >>> > >>> Thanks, > >>> -vivek > >>> > >>> On Wed, Apr 1, 2009 at 1:55 AM, Shalin Shekhar Mangar > >>> wrote: > >>>> On Wed, Apr 1, 2009 at 1:48 PM, vivek sar wrote: > >>>>> I'm using the latest released one - Solr 1.3. The wiki says passing > >>>>> dataDir to CREATE action (web service) should work, but that doesn't > >>>>> seem to be working. > >>>>> > >>>> > >>>> That is a Solr 1.4 feature (not released yet). > >>>> > >>>> -- > >>>> Regards, > >>>> Shalin Shekhar Mangar. > >>>> > >>> > >> > >> > > > > -- > > View this message in context: > http://www.nabble.com/Defining-DataDir-in-Multi-Core-tp22818543p23611179.html > > Sent from the Solr - User mailing list archive at Nabble.com. > > > > >
Re: What are the basic requirements for on-the-fly registration/creation of new Core?
Thanks Paul.
But I'm a bit confused. I've some questions.
# As you said I should have a base core instance directory containing the
conf directory, solrconfig.xml and schema.xml. Are you Before continuing
further I would like to know that the schema and solrconfig files placed
under the example core0/conf [or core1/conf] look like stripped down version
of what was available in base solrconfig and schema. So I tried replacing
the stripped down files with the base ones [commenting out the data
directory tag, as I'm using solr.xml @solr.home] for multicore. After
restarting tomcat, surprisingly the core with replaced files didn't show up
in the solr welcome page [http://localhost/solr] but the other one was there
and I moved around all the admin pages and others. If we are going to make
use of this stripped down schema and config files then what about all those
things that were mentioned in the base file[I see a lot of handlers, filter
factories and many many things], how are we going to make sure that they get
loaded when solr wakes up. Or the stripped down files are good enough. I
need proper guidance on this as I'm still learning to make use of this
wonderful tool called Solr. Thank you Solr guys.
# You said that I can put the data directory tag things in the
solrconfig.xml for solr1.3. I agree but where is the file. Are you talking
about the stripped down solrconfig. If yes then this doesnot have any data
directory entry but the base solrconf file has this entry, though. If you
meant adding the entry in the stripped one, do confirm me. I'll give it a
try.
Thanks,
KK.
2009/5/19 Noble Paul നോബിള് नोब्ळ्
> On Tue, May 19, 2009 at 2:08 PM, KK wrote:
> > Hi All,
> > Finally I made the multicore thing running by copying the core0 and core1
> > directory to $solr.home and putting a solr.xml file under the same
> > directory. A bit relieved !
> > I would like to do on-the-fly registration of core[I've provision to add
> > unique corenames everytime I register a new one] and as per the basic
> > requirements mentioned in the wiki[http://wiki.apache.org/solr/CoreAdmin]
> I
> > did the following:
> > 1. set persistent="true" and sharedLIb="lib"
> > For this I actually copied the file given in wiki and put it under
> > solr.home. Started tomcat and tried to register a new core named core3
> like
> > this[I'm running solr on 8080]
> > http://localhost:8080/solr/admin/cores?action=CREATE&name=core2
> > and the browser gave me error saying that instanceDir is missing, Agreed
> > its missing but do I need to provide the path to a new directory
> everytime a
> > register a new core or any old core'x directory will do the job? Then I
> > copied an existing directory core0 to core2 with all its /bin and /conf
> and
> > other stuffs. Then I tried to refresh the earlier page in firefox and WOW
> it
> > worked. Then to confirm that the new core details has been added to
> solr.xml
> > I cross checked the file and found a new entry for core2. Its working.
> >
> > Now let me mention what I want to do,
> > # I want multiple cores, true but all the core will essentially have the
> > same schema and config as well.
> > # I want separete index for each core so that during posting and
> searching I
> > can search a specific core
> >
> > Now I've certain doubts about these things.
> > 1. Do we have to create a new directory with all those /bin and /conf
> > everytime we've to register a new core. This sounds silly and I'm pretty
> > sure I must be wrong . There must be some way to just say that : Create
> this
> > new core having the core name as ABXZ. The wiki does say the last three
> > parameters are optinal [Wiki:Note that config ,schema & dataDir
> parameters
> > are optional.] Can't we have some base schema and config file say under
> > $solr.home [or may be somewhere else if that creates problem] and
> everytime
> > we create a new core just point to those. Can someone give me direct
> > examples for the same[the wiki does say about this but I'm not completely
> > clear about that, some examples will do the job].
> > 2. Do we have to make sure that the data directory is sitting inside
> coreX
> > directory, or we can put it elsewhere?
>
> you don't have to create conf dir and schema.xml and solrconfig.xml
> for each core separately. keep one instance dir where you have the
> conf dir and its contents. pass the instanceDir when you create a new
> core.
>
> if you are using 1.3
> in the solrconfig.xml , keep the datadir as follows
>
> /data/solr/${core.name}
>
> each core will automatically have a diffferent dataDir
>
>
>
> >
> > Thank you very much.
> > KK.
> >
>
>
>
> --
> -
> Noble Paul | Principal Engineer| AOL | http://aol.com
>
Re: What are the basic requirements for on-the-fly registration/creation of new Core?
Thank you very much Paul. The pictorial explanation was very helpful, made
everything clear. Thank you again.
AFAIU, I can put my existing schema.xml and solrconfig.xml under conf/. I
assume you also meant passing the paths to common schema and solrconfig.xml
when registering a new core? My earlier try gave me some errors when I tried
to do the same. I'll start afresh and let you know.
Thank you,
KK.
2009/5/19 Noble Paul നോബിള് नोब्ळ्
> forget about the default setup that comes w/ the example.
>
> say you have the following structure
> /data/solr/home/
> solr.xml
> conf/
>schema.xml
>solrconfig.xml
> data/
>/c1/
> index
>/c2/
> index
>/c3/
> index
>
> * your solr.solr.home is /data/solr/home
> * your common instanceDir will be /data/solr/home/ again. pass this
> parameter whenever you create a new core.
> * the conf dir contains the common files required for each core.
> * in your common solrconfig.xml keep a tag
> /data/solr/home/data/${solr.core.name} .(my
> previous mail had a mistake
> *assume your core names are c1,c2,c3 then solr will automatically
> create a structure as described above. (no need to create the dirs c1
> or c2 or c3)
>
> HTH
>
>
>
> On Tue, May 19, 2009 at 4:28 PM, KK wrote:
> > Thanks Paul.
> >
> > But I'm a bit confused. I've some questions.
> > # As you said I should have a base core instance directory containing the
> > conf directory, solrconfig.xml and schema.xml. Are you Before continuing
> > further I would like to know that the schema and solrconfig files placed
> > under the example core0/conf [or core1/conf] look like stripped down
> version
> > of what was available in base solrconfig and schema. So I tried replacing
> > the stripped down files with the base ones [commenting out the data
> > directory tag, as I'm using solr.xml @solr.home] for multicore. After
> > restarting tomcat, surprisingly the core with replaced files didn't show
> up
> > in the solr welcome page [http://localhost/solr] but the other one was
> there
> > and I moved around all the admin pages and others. If we are going to
> make
> > use of this stripped down schema and config files then what about all
> those
> > things that were mentioned in the base file[I see a lot of handlers,
> filter
> > factories and many many things], how are we going to make sure that they
> get
> > loaded when solr wakes up. Or the stripped down files are good enough. I
> > need proper guidance on this as I'm still learning to make use of this
> > wonderful tool called Solr. Thank you Solr guys.
> >
> > # You said that I can put the data directory tag things in the
> > solrconfig.xml for solr1.3. I agree but where is the file. Are you
> talking
> > about the stripped down solrconfig. If yes then this doesnot have any
> data
> > directory entry but the base solrconf file has this entry, though. If you
> > meant adding the entry in the stripped one, do confirm me. I'll give it a
> > try.
> >
> > Thanks,
> > KK.
> >
> > 2009/5/19 Noble Paul നോബിള് नोब्ळ्
> >
> >> On Tue, May 19, 2009 at 2:08 PM, KK wrote:
> >> > Hi All,
> >> > Finally I made the multicore thing running by copying the core0 and
> core1
> >> > directory to $solr.home and putting a solr.xml file under the same
> >> > directory. A bit relieved !
> >> > I would like to do on-the-fly registration of core[I've provision to
> add
> >> > unique corenames everytime I register a new one] and as per the basic
> >> > requirements mentioned in the wiki[
> http://wiki.apache.org/solr/CoreAdmin]
> >> I
> >> > did the following:
> >> > 1. set persistent="true" and sharedLIb="lib"
> >> > For this I actually copied the file given in wiki and put it under
> >> > solr.home. Started tomcat and tried to register a new core named core3
> >> like
> >> > this[I'm running solr on 8080]
> >> > http://localhost:8080/solr/admin/cores?action=CREATE&name=core2
> >> > and the browser gave me error saying that instanceDir is missing,
> Agreed
> >> > its missing but do I need to provide the path to a new directory
> >> everytime a
> >> > register a new core or any old
Re: What are the basic requirements for on-the-fly registration/creation of new Core?
Thanks a lot @Paul.
A did as you said and added an entry in solrconfig.xml for data directory.
But when I tried to register an new core it gave me the following error. It
seems its not able to pick the core name from ${solr.core.name} [even tried
what you mentioned in your last mail, ${core.name} but no luck.
HTTP Status 500 - No system property or default value specified for
core.name org.apache.solr.common.SolrException: No system property or
default value specified for solr.core.name at
org.apache.solr.common.util.DOMUtil.substituteProperty(DOMUtil.java:311) at
org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:264)
at
org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:272)
at
org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:272)
at org.apache.solr.core.Config.(Config.java:105) at
org.apache.solr.core.SolrConfig.(SolrConfig.java:113) at
org.apache.solr.core.CoreContainer.create(CoreContainer.java:321) at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:107)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875)
at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
at java.lang.Thread.run(Thread.java:619)
Thanks,
KK.
2009/5/19 Noble Paul നോബിള് नोब्ळ्
> On Tue, May 19, 2009 at 5:32 PM, KK wrote:
> > Thank you very much Paul. The pictorial explanation was very helpful,
> made
> > everything clear. Thank you again.
> > AFAIU, I can put my existing schema.xml and solrconfig.xml under conf/. I
> > assume you also meant passing the paths to common schema and
> solrconfig.xml
> > when registering a new core? My earlier try gave me some errors when I
> tried
> > to do the same. I'll start afresh and let you know.
> nope the instanceDir is the only attribute you need to pass while
> creating the core
> >
> > Thank you,
> > KK.
> >
> > 2009/5/19 Noble Paul നോബിള് नोब्ळ्
> >
> >> forget about the default setup that comes w/ the example.
> >>
> >> say you have the following structure
> >> /data/solr/home/
> >> solr.xml
> >> conf/
> >>schema.xml
> >>solrconfig.xml
> >> data/
> >>/c1/
> >> index
> >>/c2/
> >> index
> >>/c3/
> >> index
> >>
> >> * your solr.solr.home is /data/solr/home
> >> * your common instanceDir will be /data/solr/home/ again. pass this
> >> parameter whenever you create a new core.
> >> * the conf dir contains the common files required for each core.
> >> * in your common solrconfig.xml keep a tag
> >> /data/solr/home/data/${solr.core.name} .(my
> >> previous mail had a mistake
> >> *assume your core names are c1,c2,c3 then solr will automatically
> >> create a structure as described above. (no need to create the dirs c1
> >> or c2 or c3)
> >>
> >> HTH
> >>
> >>
> >>
> >> On Tue, May 19, 2009 at 4:28 PM, KK wrote:
> >> > Thanks Paul.
> >> >
> >> > But I'm a bit confused. I've some questions.
> >> > # As you said I shoul
Re: What are the basic requirements for on-the-fly registration/creation of new Core?
I'm using Solr1.3. I installed it two weeks ago. Current version is 1.3, I
guess? Where to get 1.4
Do I've to use solr1.4 dev[is it available for public use?] or any patch to
fix the bug?
Thanks,
KK.
2009/5/19 Noble Paul നോബിള് नोब्ळ्
> which build are you using ? I guess Solr1.3 had a problem with setting
> implicit properties which is fixed in Solr 1.4
>
> https://issues.apache.org/jira/browse/SOLR-883
>
>
>
> On Tue, May 19, 2009 at 6:16 PM, KK wrote:
> > Thanks a lot @Paul.
> > A did as you said and added an entry in solrconfig.xml for data
> directory.
> > But when I tried to register an new core it gave me the following error.
> It
> > seems its not able to pick the core name from ${solr.core.name} [even
> tried
> > what you mentioned in your last mail, ${core.name} but no luck.
> >
> >
> > HTTP Status 500 - No system property or default value specified for
> > core.name org.apache.solr.common.SolrException: No system property or
> > default value specified for solr.core.name at
> > org.apache.solr.common.util.DOMUtil.substituteProperty(DOMUtil.java:311)
> at
> >
> org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:264)
> > at
> >
> org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:272)
> > at
> >
> org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:272)
> > at org.apache.solr.core.Config.(Config.java:105) at
> > org.apache.solr.core.SolrConfig.(SolrConfig.java:113) at
> > org.apache.solr.core.CoreContainer.create(CoreContainer.java:321) at
> >
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:107)
> > at
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) at
> >
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
> > at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
> > at
> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
> > at
> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> > at
> >
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
> > at
> >
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
> > at
> >
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> > at
> >
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
> > at
> >
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
> > at
> >
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
> > at
> >
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875)
> > at
> >
> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
> > at
> >
> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
> > at
> >
> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
> > at
> >
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
> > at java.lang.Thread.run(Thread.java:619)
> >
> >
> > Thanks,
> > KK.
> >
> >
> > 2009/5/19 Noble Paul നോബിള് नोब्ळ्
> >>
> >> On Tue, May 19, 2009 at 5:32 PM, KK wrote:
> >> > Thank you very much Paul. The pictorial explanation was very helpful,
> >> > made
> >> > everything clear. Thank you again.
> >> > AFAIU, I can put my existing schema.xml and solrconfig.xml under
> conf/.
> >> > I
> >> > assume you also meant passing the paths to common schema and
> >> > solrconfig.xml
> >> > when registering a new core? My earlier try gave me some errors when I
> >> > tried
> >> > to do the same. I'll start afresh and let you know.
> >> nope the instanceDir is the only attribute you need to pass while
> >> creating the core
> >> >
> >> > Thank you,
> >> > KK.
> >> >
> >> > 2009/5/19 Noble Paul നോബിള് नोब्ळ्
> >> >
> >> >> forget about the default setup that comes w/ the example.
> >> >>
> >> >> say you have the following structure
> >> >> /data/solr/h
Re: What are the basic requirements for on-the-fly registration/creation of new Core?
That jira mentions a patch. That will fix the bug, right?.
How do I apply the patch?
Thanks,
KK>
On Tue, May 19, 2009 at 7:52 PM, KK wrote:
> I'm using Solr1.3. I installed it two weeks ago. Current version is 1.3, I
> guess? Where to get 1.4
> Do I've to use solr1.4 dev[is it available for public use?] or any patch to
> fix the bug?
>
>
> Thanks,
> KK.
>
> 2009/5/19 Noble Paul നോബിള് नोब्ळ्
>
>> which build are you using ? I guess Solr1.3 had a problem with setting
>> implicit properties which is fixed in Solr 1.4
>>
>> https://issues.apache.org/jira/browse/SOLR-883
>>
>>
>>
>> On Tue, May 19, 2009 at 6:16 PM, KK wrote:
>> > Thanks a lot @Paul.
>> > A did as you said and added an entry in solrconfig.xml for data
>> directory.
>> > But when I tried to register an new core it gave me the following error.
>> It
>> > seems its not able to pick the core name from ${solr.core.name} [even
>> tried
>> > what you mentioned in your last mail, ${core.name} but no luck.
>> >
>> >
>> > HTTP Status 500 - No system property or default value specified for
>> > core.name org.apache.solr.common.SolrException: No system property or
>> > default value specified for solr.core.name at
>> > org.apache.solr.common.util.DOMUtil.substituteProperty(DOMUtil.java:311)
>> at
>> >
>> org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:264)
>> > at
>> >
>> org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:272)
>> > at
>> >
>> org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:272)
>> > at org.apache.solr.core.Config.(Config.java:105) at
>> > org.apache.solr.core.SolrConfig.(SolrConfig.java:113) at
>> > org.apache.solr.core.CoreContainer.create(CoreContainer.java:321) at
>> >
>> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:107)
>> > at
>> >
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
>> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) at
>> >
>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
>> > at
>> >
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
>> > at
>> >
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
>> > at
>> >
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
>> > at
>> >
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
>> > at
>> >
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
>> > at
>> >
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>> > at
>> >
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
>> > at
>> >
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
>> > at
>> >
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
>> > at
>> >
>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875)
>> > at
>> >
>> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
>> > at
>> >
>> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
>> > at
>> >
>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
>> > at
>> >
>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
>> > at java.lang.Thread.run(Thread.java:619)
>> >
>> >
>> > Thanks,
>> > KK.
>> >
>> >
>> > 2009/5/19 Noble Paul നോബിള് नोब्ळ्
>> >>
>> >> On Tue, May 19, 2009 at 5:32 PM, KK
>> wrote:
>> >> > Thank you very much Paul. The pictorial explanation was very helpful,
>> >> > made
>> >> > everything clear. Thank you again.
>> >> > AFAIU, I can put my existing schema.xml and solrconfig.xml under
>> conf/.
>> >> > I
>> >> > assume you also meant passing the paths to common schema and
>> >> > solrconfig.xml
>&
Re: What are the basic requirements for on-the-fly registration/creation of new Core?
I'm downloading the nightly build (19thMay09, why is the size so huge ~128
MB whereas the normal gz one is ~21 MB). I think I just have to replace the
solr.war file with the nightly one, under tomcat and my rest of the config
files in $solr.home will remain same. Correct me if I'm wrong. Nightly build
are the latest ones, right? with all bug fixes done so far included in that
build.
Thanks,
KK.
2009/5/19 Noble Paul നോബിള് नोब्ळ्
> I am not sure whether it will apply on the 1.3 source. you can give it a
> try.
>
> .a nightly build is your best bet.
>
>
>
> On Tue, May 19, 2009 at 7:54 PM, KK wrote:
> > That jira mentions a patch. That will fix the bug, right?.
> > How do I apply the patch?
> >
> > Thanks,
> > KK>
> >
> > On Tue, May 19, 2009 at 7:52 PM, KK wrote:
> >>
> >> I'm using Solr1.3. I installed it two weeks ago. Current version is 1.3,
> I
> >> guess? Where to get 1.4
> >> Do I've to use solr1.4 dev[is it available for public use?] or any patch
> >> to fix the bug?
> >>
> >> Thanks,
> >> KK.
> >>
> >> 2009/5/19 Noble Paul നോബിള് नोब्ळ्
> >>>
> >>> which build are you using ? I guess Solr1.3 had a problem with setting
> >>> implicit properties which is fixed in Solr 1.4
> >>>
> >>> https://issues.apache.org/jira/browse/SOLR-883
> >>>
> >>>
> >>>
> >>> On Tue, May 19, 2009 at 6:16 PM, KK
> wrote:
> >>> > Thanks a lot @Paul.
> >>> > A did as you said and added an entry in solrconfig.xml for data
> >>> > directory.
> >>> > But when I tried to register an new core it gave me the following
> >>> > error. It
> >>> > seems its not able to pick the core name from ${solr.core.name}
> [even
> >>> > tried
> >>> > what you mentioned in your last mail, ${core.name} but no luck.
> >>> >
> >>> >
> >>> > HTTP Status 500 - No system property or default value specified for
> >>> > core.name org.apache.solr.common.SolrException: No system property
> or
> >>> > default value specified for solr.core.name at
> >>> >
> >>> >
> org.apache.solr.common.util.DOMUtil.substituteProperty(DOMUtil.java:311) at
> >>> >
> >>> >
> org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:264)
> >>> > at
> >>> >
> >>> >
> org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:272)
> >>> > at
> >>> >
> >>> >
> org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:272)
> >>> > at org.apache.solr.core.Config.(Config.java:105) at
> >>> > org.apache.solr.core.SolrConfig.(SolrConfig.java:113) at
> >>> > org.apache.solr.core.CoreContainer.create(CoreContainer.java:321) at
> >>> >
> >>> >
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:107)
> >>> > at
> >>> >
> >>> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> >>> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) at
> >>> >
> >>> >
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
> >>> > at
> >>> >
> >>> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
> >>> > at
> >>> >
> >>> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
> >>> > at
> >>> >
> >>> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> >>> > at
> >>> >
> >>> >
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
> >>> > at
> >>> >
> >>> >
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
> >>> > at
> >>> >
> >>> >
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> >>> > at
> >>> >
> >>> >
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
> >>> > at
> >>> >
> >>> >
> org.apache.catalina.core
Re: What are the basic requirements for on-the-fly registration/creation of new Core?
Thank you. Let me try the nightly build then I'll let you know.
Thanks,
KK.
2009/5/20 Noble Paul നോബിള് नोब्ळ्
> The dowload is big because it contains a lot of things (read contribs)
> which you may not need. You can just take the solr.war from that.
>
> everything else remains same. All the conf files should be completely
> backward compatible.
>
> all resolved issues will be available in the nightly
>
>
> On Wed, May 20, 2009 at 10:19 AM, KK wrote:
> > I'm downloading the nightly build (19thMay09, why is the size so huge
> ~128
> > MB whereas the normal gz one is ~21 MB). I think I just have to replace
> the
> > solr.war file with the nightly one, under tomcat and my rest of the
> config
> > files in $solr.home will remain same. Correct me if I'm wrong. Nightly
> build
> > are the latest ones, right? with all bug fixes done so far included in
> that
> > build.
> >
> >
> > Thanks,
> > KK.
> >
> > 2009/5/19 Noble Paul നോബിള് नोब्ळ्
> >
> >> I am not sure whether it will apply on the 1.3 source. you can give it a
> >> try.
> >>
> >> .a nightly build is your best bet.
> >>
> >>
> >>
> >> On Tue, May 19, 2009 at 7:54 PM, KK wrote:
> >> > That jira mentions a patch. That will fix the bug, right?.
> >> > How do I apply the patch?
> >> >
> >> > Thanks,
> >> > KK>
> >> >
> >> > On Tue, May 19, 2009 at 7:52 PM, KK
> wrote:
> >> >>
> >> >> I'm using Solr1.3. I installed it two weeks ago. Current version is
> 1.3,
> >> I
> >> >> guess? Where to get 1.4
> >> >> Do I've to use solr1.4 dev[is it available for public use?] or any
> patch
> >> >> to fix the bug?
> >> >>
> >> >> Thanks,
> >> >> KK.
> >> >>
> >> >> 2009/5/19 Noble Paul നോബിള് नोब्ळ्
> >> >>>
> >> >>> which build are you using ? I guess Solr1.3 had a problem with
> setting
> >> >>> implicit properties which is fixed in Solr 1.4
> >> >>>
> >> >>> https://issues.apache.org/jira/browse/SOLR-883
> >> >>>
> >> >>>
> >> >>>
> >> >>> On Tue, May 19, 2009 at 6:16 PM, KK
> >> wrote:
> >> >>> > Thanks a lot @Paul.
> >> >>> > A did as you said and added an entry in solrconfig.xml for data
> >> >>> > directory.
> >> >>> > But when I tried to register an new core it gave me the following
> >> >>> > error. It
> >> >>> > seems its not able to pick the core name from ${solr.core.name}
> >> [even
> >> >>> > tried
> >> >>> > what you mentioned in your last mail, ${core.name} but no luck.
> >> >>> >
> >> >>> >
> >> >>> > HTTP Status 500 - No system property or default value specified
> for
> >> >>> > core.name org.apache.solr.common.SolrException: No system
> property
> >> or
> >> >>> > default value specified for solr.core.name at
> >> >>> >
> >> >>> >
> >> org.apache.solr.common.util.DOMUtil.substituteProperty(DOMUtil.java:311)
> at
> >> >>> >
> >> >>> >
> >>
> org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:264)
> >> >>> > at
> >> >>> >
> >> >>> >
> >>
> org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:272)
> >> >>> > at
> >> >>> >
> >> >>> >
> >>
> org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:272)
> >> >>> > at org.apache.solr.core.Config.(Config.java:105) at
> >> >>> > org.apache.solr.core.SolrConfig.(SolrConfig.java:113) at
> >> >>> > org.apache.solr.core.CoreContainer.create(CoreContainer.java:321)
> at
> >> >>> >
> >> >>> >
> >>
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:107)
> >> >>> > at
> >> >>> >
> >> >>> >
> >>
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> >> >>> > at org.apache.solr.cor
Re: What are the basic requirements for on-the-fly registration/creation of new Core?
I downloaded the nightly build and deployed the dist/solr-nightly-build.war in tomcat with all other config same @solr.home. Trying to create a new core gave me the same error. Do I still need to apply the patch you mentioned earlier. I'm stuck, please help me out. I used this to create a new core, http://localhost:8080/solr/admin/cores?action=CREATE&name=core2&instanceDir=/opt/solr/ my solr home is /opt/solr which has a conf directory containing solrconfig and schema files. Thanks, KK *message* *No system property or default value specified for solr.core.nameorg.apache.solr.common.SolrException: No system property or default value specified for solr.core.name at org.apache.solr.common.util.DOMUtil.substituteProperty(DOMUtil.java:311) at org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:264) at org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:272) at org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:272) at org.apache.solr.core.Config.(Config.java:105) at org.apache.solr.core.SolrConfig.(SolrConfig.java:113) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:321) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:107) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875) at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665) at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528) at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689) at java.lang.Thread.run(Thread.java:619) * *description* *The server encountered an internal error (No system property or default value specified for solr.core.nameorg.apache.solr.common.SolrException: No system property or default value specified for solr.core.name at org.apache.solr.common.util.DOMUtil.substituteProperty(DOMUtil.java:311) at org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:264) at org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:272) at org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:272) at org.apache.solr.core.Config.(Config.java:105) at org.apache.solr.core.SolrConfig.(SolrConfig.java:113) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:321) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:107) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875) at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665) at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528) at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThrea
Re: What are the basic requirements for on-the-fly registration/creation of new Core?
Please do so ASAP. I'm stuck just becaue of that. If not then I might have to move to lucene. I guess lucene has support for on the fly creation of new indexes. Any idea on this? Thanks, KK. 2009/5/20 Noble Paul നോബിള് नोब्ळ् > hi KK, > the fix got removed in a subsequent refactoring I guess. I shall check > in soon and you can get it from the trunk. > > On Wed, May 20, 2009 at 11:29 AM, KK wrote: > > I downloaded the nightly build and deployed the > dist/solr-nightly-build.war > > in tomcat with all other config same @solr.home. Trying to create a new > core > > gave me the same error. Do I still need to apply the patch you mentioned > > earlier. I'm stuck, please help me out. > > I used this to create a new core, > > > http://localhost:8080/solr/admin/cores?action=CREATE&name=core2&instanceDir=/opt/solr/ > > > > my solr home is /opt/solr which has a conf directory containing > solrconfig > > and schema files. > > > > Thanks, > > KK > > > > message No system property or default value specified for solr.core.name > > org.apache.solr.common.SolrException: No system property or default value > > specified for solr.core.name at > > org.apache.solr.common.util.DOMUtil.substituteProperty(DOMUtil.java:311) > at > > > org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:264) > > at > > > org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:272) > > at > > > org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:272) > > at org.apache.solr.core.Config.(Config.java:105) at > > org.apache.solr.core.SolrConfig.(SolrConfig.java:113) at > > org.apache.solr.core.CoreContainer.create(CoreContainer.java:321) at > > > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:107) > > at > > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) > > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) at > > > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303) > > at > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232) > > at > > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) > > at > > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) > > at > > > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) > > at > > > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172) > > at > > > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) > > at > > > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) > > at > > > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108) > > at > > > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174) > > at > > > org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875) > > at > > > org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665) > > at > > > org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528) > > at > > > org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81) > > at > > > org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689) > > at java.lang.Thread.run(Thread.java:619) > > > > description The server encountered an internal error (No system property > or > > default value specified for solr.core.name > > org.apache.solr.common.SolrException: No system property or default value > > specified for solr.core.name at > > org.apache.solr.common.util.DOMUtil.substituteProperty(DOMUtil.java:311) > at > > > org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:264) > > at > > > org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:272) > > at > > > org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:272) > > at org.apache.solr.core.Config.(Config.java:105) at > > org.apache.solr.core.SolrConfig.(SolrConfig.java:113) at > > org.apache.solr.core.CoreContainer.create(CoreContainer.java:321) at > > > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:107) > > at > > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHa
Re: What are the basic requirements for on-the-fly registration/creation of new Core?
I downloaded the nightly build . How do I get the trunk and where to get it from? I've never used trunks earlier . Please help me out. Tell me the detailed steps for all this. Thanks, KK. 2009/5/20 Noble Paul നോബിള് नोब्ळ् > if you are using a trunk version you can still use the dataDir > attribute while creating the core. so you are not really stuck > > I have committed the fix > > On Wed, May 20, 2009 at 11:45 AM, KK wrote: > > Please do so ASAP. I'm stuck just becaue of that. If not then I might > have > > to move to lucene. I guess lucene has support for on the fly creation of > new > > indexes. Any idea on this? > > > > Thanks, > > KK. > > > > 2009/5/20 Noble Paul നോബിള് नोब्ळ् > >> > >> hi KK, > >> the fix got removed in a subsequent refactoring I guess. I shall check > >> in soon and you can get it from the trunk. > >> > >> On Wed, May 20, 2009 at 11:29 AM, KK > wrote: > >> > I downloaded the nightly build and deployed the > >> > dist/solr-nightly-build.war > >> > in tomcat with all other config same @solr.home. Trying to create a > new > >> > core > >> > gave me the same error. Do I still need to apply the patch you > mentioned > >> > earlier. I'm stuck, please help me out. > >> > I used this to create a new core, > >> > > >> > > http://localhost:8080/solr/admin/cores?action=CREATE&name=core2&instanceDir=/opt/solr/ > >> > > >> > my solr home is /opt/solr which has a conf directory containing > >> > solrconfig > >> > and schema files. > >> > > >> > Thanks, > >> > KK > >> > > >> > message No system property or default value specified for > solr.core.name > >> > org.apache.solr.common.SolrException: No system property or default > >> > value > >> > specified for solr.core.name at > >> > > org.apache.solr.common.util.DOMUtil.substituteProperty(DOMUtil.java:311) > >> > at > >> > > >> > > org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:264) > >> > at > >> > > >> > > org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:272) > >> > at > >> > > >> > > org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:272) > >> > at org.apache.solr.core.Config.(Config.java:105) at > >> > org.apache.solr.core.SolrConfig.(SolrConfig.java:113) at > >> > org.apache.solr.core.CoreContainer.create(CoreContainer.java:321) at > >> > > >> > > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:107) > >> > at > >> > > >> > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) > >> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) at > >> > > >> > > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303) > >> > at > >> > > >> > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232) > >> > at > >> > > >> > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) > >> > at > >> > > >> > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) > >> > at > >> > > >> > > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) > >> > at > >> > > >> > > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172) > >> > at > >> > > >> > > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) > >> > at > >> > > >> > > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) > >> > at > >> > > >> > > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108) > >> > at > >> > > >> > > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174) > >> > at > >> > > >> > > org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875) > >> > at > >> > > >> > > org.apache.coyote.http11.Http11BaseProtocol$Http11Conne
Re: What are the basic requirements for on-the-fly registration/creation of new Core?
Do I've to check out this http://svn.apache.org/repos/asf/lucene/solr/trunk/ using this command svn checkout http://svn.apache.org/repos/asf/lucene/solr/trunk/ ??? not much idea about svn chekout from public repo. give me some specific pointers on this. and then fire ant to build a fresh solr? Will that create the war to be deployed or I've to create the war file for deployment? please give me proper directions. Thanks, KK. 2009/5/20 Noble Paul നോബിള് नोब्ळ् > you will have to wait another 3 hours for the next nightly build to happen. > > you can of course check out the trunk and build one your own > > see this > > http://lucene.apache.org/solr/version_control.html#Anonymous+Access+(read-only)<http://lucene.apache.org/solr/version_control.html#Anonymous+Access+%28read-only%29> > > now you may need ant to build this. > > On Wed, May 20, 2009 at 11:53 AM, KK wrote: > > I downloaded the nightly build . How do I get the trunk and where to get > it > > from? I've never used trunks earlier . Please help me out. Tell me the > > detailed steps for all this. > > > > Thanks, > > KK. > > > > 2009/5/20 Noble Paul നോബിള് नोब्ळ् > >> > >> if you are using a trunk version you can still use the dataDir > >> attribute while creating the core. so you are not really stuck > >> > >> I have committed the fix > >> > >> On Wed, May 20, 2009 at 11:45 AM, KK > wrote: > >> > Please do so ASAP. I'm stuck just becaue of that. If not then I might > >> > have > >> > to move to lucene. I guess lucene has support for on the fly creation > of > >> > new > >> > indexes. Any idea on this? > >> > > >> > Thanks, > >> > KK. > >> > > >> > 2009/5/20 Noble Paul നോബിള് नोब्ळ् > >> >> > >> >> hi KK, > >> >> the fix got removed in a subsequent refactoring I guess. I shall > check > >> >> in soon and you can get it from the trunk. > >> >> > >> >> On Wed, May 20, 2009 at 11:29 AM, KK > >> >> wrote: > >> >> > I downloaded the nightly build and deployed the > >> >> > dist/solr-nightly-build.war > >> >> > in tomcat with all other config same @solr.home. Trying to create a > >> >> > new > >> >> > core > >> >> > gave me the same error. Do I still need to apply the patch you > >> >> > mentioned > >> >> > earlier. I'm stuck, please help me out. > >> >> > I used this to create a new core, > >> >> > > >> >> > > >> >> > > http://localhost:8080/solr/admin/cores?action=CREATE&name=core2&instanceDir=/opt/solr/ > >> >> > > >> >> > my solr home is /opt/solr which has a conf directory containing > >> >> > solrconfig > >> >> > and schema files. > >> >> > > >> >> > Thanks, > >> >> > KK > >> >> > > >> >> > message No system property or default value specified for > >> >> > solr.core.name > >> >> > org.apache.solr.common.SolrException: No system property or default > >> >> > value > >> >> > specified for solr.core.name at > >> >> > > >> >> > > org.apache.solr.common.util.DOMUtil.substituteProperty(DOMUtil.java:311) > >> >> > at > >> >> > > >> >> > > >> >> > > org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:264) > >> >> > at > >> >> > > >> >> > > >> >> > > org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:272) > >> >> > at > >> >> > > >> >> > > >> >> > > org.apache.solr.common.util.DOMUtil.substituteProperties(DOMUtil.java:272) > >> >> > at org.apache.solr.core.Config.(Config.java:105) at > >> >> > org.apache.solr.core.SolrConfig.(SolrConfig.java:113) at > >> >> > org.apache.solr.core.CoreContainer.create(CoreContainer.java:321) > at > >> >> > > >> >> > > >> >> > > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:107) > >> >> > at > >> >> > > >> >> > > >> >> >
Re: What are the basic requirements for on-the-fly registration/creation of new Core?
OK then , I assume that nightly build will solve my basic problem of "On the fly creation of new cores using dataDir as req parameter", then I can wait for two more hours. One more thing the new nightly build willl be called solr-2009-05-20.tgz, right as teh current one is solr-2009-05-19.tgz, right? Thanks, KK. 2009/5/20 Noble Paul നോബിള് नोब्ळ् > if you are not familiar w/ svn and ant I suggest wait till the nightly > build happens it is just 2 hrs away > > > On Wed, May 20, 2009 at 12:13 PM, KK wrote: > > Do I've to check out this > > http://svn.apache.org/repos/asf/lucene/solr/trunk/ > > using this command > > svn checkout http://svn.apache.org/repos/asf/lucene/solr/trunk/ > > ??? not much idea about svn chekout from public repo. give me some > specific > > pointers on this. > > > > and then fire ant to build a fresh solr? Will that create the war to be > > deployed or I've to create the war file for deployment? please give me > > proper directions. > > > > Thanks, > > KK. > > > > 2009/5/20 Noble Paul നോബിള് नोब्ळ् > >> > >> you will have to wait another 3 hours for the next nightly build to > >> happen. > >> > >> you can of course check out the trunk and build one your own > >> > >> see this > >> > >> > http://lucene.apache.org/solr/version_control.html#Anonymous+Access+(read-only)<http://lucene.apache.org/solr/version_control.html#Anonymous+Access+%28read-only%29> > >> > >> now you may need ant to build this. > >> > >> On Wed, May 20, 2009 at 11:53 AM, KK > wrote: > >> > I downloaded the nightly build . How do I get the trunk and where to > get > >> > it > >> > from? I've never used trunks earlier . Please help me out. Tell me the > >> > detailed steps for all this. > >> > > >> > Thanks, > >> > KK. > >> > > >> > 2009/5/20 Noble Paul നോബിള് नोब्ळ् > >> >> > >> >> if you are using a trunk version you can still use the dataDir > >> >> attribute while creating the core. so you are not really stuck > >> >> > >> >> I have committed the fix > >> >> > >> >> On Wed, May 20, 2009 at 11:45 AM, KK > >> >> wrote: > >> >> > Please do so ASAP. I'm stuck just becaue of that. If not then I > might > >> >> > have > >> >> > to move to lucene. I guess lucene has support for on the fly > creation > >> >> > of > >> >> > new > >> >> > indexes. Any idea on this? > >> >> > > >> >> > Thanks, > >> >> > KK. > >> >> > > >> >> > 2009/5/20 Noble Paul നോബിള് नोब्ळ् > >> >> >> > >> >> >> hi KK, > >> >> >> the fix got removed in a subsequent refactoring I guess. I shall > >> >> >> check > >> >> >> in soon and you can get it from the trunk. > >> >> >> > >> >> >> On Wed, May 20, 2009 at 11:29 AM, KK > >> >> >> wrote: > >> >> >> > I downloaded the nightly build and deployed the > >> >> >> > dist/solr-nightly-build.war > >> >> >> > in tomcat with all other config same @solr.home. Trying to > create > >> >> >> > a > >> >> >> > new > >> >> >> > core > >> >> >> > gave me the same error. Do I still need to apply the patch you > >> >> >> > mentioned > >> >> >> > earlier. I'm stuck, please help me out. > >> >> >> > I used this to create a new core, > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > > http://localhost:8080/solr/admin/cores?action=CREATE&name=core2&instanceDir=/opt/solr/ > >> >> >> > > >> >> >> > my solr home is /opt/solr which has a conf directory containing > >> >> >> > solrconfig > >> >> >> > and schema files. > >> >> >> > > >> >> >> > Thanks, > >> >> >> > KK > >> >> >> > > >> >> >> > message No system property or default value specified for > >> >> >> > solr.core.name &g
Re: What are the basic requirements for on-the-fly registration/creation of new Core?
Hi Noble,
I downloaded the latest nightly build(20th may) and deployed the
nightly-build.war and tried to run the same thing for creating a new core
and the bad news is that it didn't work. I tried this
/opt/solr/data/${core.name} also
/opt/solr/data/${solr.core.name} but it gave me the same
error saying
HTTP Status 500 - No system property or default value specified for
core.name org.apache.solr.common.SolrException: No system property or
default value specified for core.name at
org.apache.solr.common.util.DOMUtil.substituteProperty(DOMUtil.java:311) at
...
Can you please help me out. The config files are the old ones that were
there solrconfig.xml and schema.xml withe an entry for data directory as
you've told me earlier. Do I've to pick the new solrconfig files. I'm still
stuck...
Thanks,
KK.
On Wed, May 20, 2009 at 12:59 PM, KK wrote:
> OK then , I assume that nightly build will solve my basic problem of "On
> the fly creation of new cores using dataDir as req parameter", then I can
> wait for two more hours.
> One more thing the new nightly build willl be called solr-2009-05-20.tgz,
> right as teh current one is solr-2009-05-19.tgz, right?
>
>
> Thanks,
> KK.
>
> 2009/5/20 Noble Paul നോബിള് नोब्ळ्
>
>> if you are not familiar w/ svn and ant I suggest wait till the nightly
>> build happens it is just 2 hrs away
>>
>>
>> On Wed, May 20, 2009 at 12:13 PM, KK wrote:
>> > Do I've to check out this
>> > http://svn.apache.org/repos/asf/lucene/solr/trunk/
>> > using this command
>> > svn checkout http://svn.apache.org/repos/asf/lucene/solr/trunk/
>> > ??? not much idea about svn chekout from public repo. give me some
>> specific
>> > pointers on this.
>> >
>> > and then fire ant to build a fresh solr? Will that create the war to be
>> > deployed or I've to create the war file for deployment? please give me
>> > proper directions.
>> >
>> > Thanks,
>> > KK.
>> >
>> > 2009/5/20 Noble Paul നോബിള് नोब्ळ्
>> >>
>> >> you will have to wait another 3 hours for the next nightly build to
>> >> happen.
>> >>
>> >> you can of course check out the trunk and build one your own
>> >>
>> >> see this
>> >>
>> >>
>> http://lucene.apache.org/solr/version_control.html#Anonymous+Access+(read-only)<http://lucene.apache.org/solr/version_control.html#Anonymous+Access+%28read-only%29>
>> >>
>> >> now you may need ant to build this.
>> >>
>> >> On Wed, May 20, 2009 at 11:53 AM, KK
>> wrote:
>> >> > I downloaded the nightly build . How do I get the trunk and where to
>> get
>> >> > it
>> >> > from? I've never used trunks earlier . Please help me out. Tell me
>> the
>> >> > detailed steps for all this.
>> >> >
>> >> > Thanks,
>> >> > KK.
>> >> >
>> >> > 2009/5/20 Noble Paul നോബിള് नोब्ळ्
>> >> >>
>> >> >> if you are using a trunk version you can still use the dataDir
>> >> >> attribute while creating the core. so you are not really stuck
>> >> >>
>> >> >> I have committed the fix
>> >> >>
>> >> >> On Wed, May 20, 2009 at 11:45 AM, KK
>> >> >> wrote:
>> >> >> > Please do so ASAP. I'm stuck just becaue of that. If not then I
>> might
>> >> >> > have
>> >> >> > to move to lucene. I guess lucene has support for on the fly
>> creation
>> >> >> > of
>> >> >> > new
>> >> >> > indexes. Any idea on this?
>> >> >> >
>> >> >> > Thanks,
>> >> >> > KK.
>> >> >> >
>> >> >> > 2009/5/20 Noble Paul നോബിള് नोब्ळ्
>> >> >> >>
>> >> >> >> hi KK,
>> >> >> >> the fix got removed in a subsequent refactoring I guess. I shall
>> >> >> >> check
>> >> >> >> in soon and you can get it from the trunk.
>> >> >> >>
>> >> >> >> On Wed, May 20, 2009 at 11:29 AM, KK > >
>> >> >> >> wrote:
>> >> >> >> > I downloaded the nightly build and deployed the
>> >> >> >> > dist/solr-nightly-build.war
>> >> >> >&
Re: What are the basic requirements for on-the-fly registration/creation of new Core?
Thanks for the response.
That means I've to have the directory before I pass it to solr, its not
going to create it by itself. Or just passing the name will make it create a
new directory? I've to give the full path? Seems I wont be able to register
a new core on the fly by just passing the name. Do confirm me.
Thanks,
KK.
2009/5/20 Noble Paul നോബിള് नोब्ळ्
> hi KK,
> instead of keeping the in the solrconfig.xml you can pass it
> as a parameter while creating the core.
>
> you can use that for the moment and we can test the other on eand let u
> know
>
>
> On Wed, May 20, 2009 at 3:10 PM, KK wrote:
> > Hi Noble,
> > I downloaded the latest nightly build(20th may) and deployed the
> > nightly-build.war and tried to run the same thing for creating a new core
> > and the bad news is that it didn't work. I tried this
> > /opt/solr/data/${core.name} also
> > /opt/solr/data/${solr.core.name} but it gave me the
> same
> > error saying
> >
> > HTTP Status 500 - No system property or default value specified for
> > core.name org.apache.solr.common.SolrException: No system property or
> > default value specified for core.name at
> > org.apache.solr.common.util.DOMUtil.substituteProperty(DOMUtil.java:311)
> at
> > ...
> >
> > Can you please help me out. The config files are the old ones that were
> > there solrconfig.xml and schema.xml withe an entry for data directory as
> > you've told me earlier. Do I've to pick the new solrconfig files. I'm
> still
> > stuck...
> >
> > Thanks,
> > KK.
> >
> > On Wed, May 20, 2009 at 12:59 PM, KK wrote:
> >>
> >> OK then , I assume that nightly build will solve my basic problem of "On
> >> the fly creation of new cores using dataDir as req parameter", then I
> can
> >> wait for two more hours.
> >> One more thing the new nightly build willl be called
> solr-2009-05-20.tgz,
> >> right as teh current one is solr-2009-05-19.tgz, right?
> >>
> >> Thanks,
> >> KK.
> >>
> >> 2009/5/20 Noble Paul നോബിള് नोब्ळ्
> >>>
> >>> if you are not familiar w/ svn and ant I suggest wait till the nightly
> >>> build happens it is just 2 hrs away
> >>>
> >>>
> >>> On Wed, May 20, 2009 at 12:13 PM, KK
> wrote:
> >>> > Do I've to check out this
> >>> > http://svn.apache.org/repos/asf/lucene/solr/trunk/
> >>> > using this command
> >>> > svn checkout http://svn.apache.org/repos/asf/lucene/solr/trunk/
> >>> > ??? not much idea about svn chekout from public repo. give me some
> >>> > specific
> >>> > pointers on this.
> >>> >
> >>> > and then fire ant to build a fresh solr? Will that create the war to
> be
> >>> > deployed or I've to create the war file for deployment? please give
> me
> >>> > proper directions.
> >>> >
> >>> > Thanks,
> >>> > KK.
> >>> >
> >>> > 2009/5/20 Noble Paul നോബിള് नोब्ळ्
> >>> >>
> >>> >> you will have to wait another 3 hours for the next nightly build to
> >>> >> happen.
> >>> >>
> >>> >> you can of course check out the trunk and build one your own
> >>> >>
> >>> >> see this
> >>> >>
> >>> >>
> >>> >>
> http://lucene.apache.org/solr/version_control.html#Anonymous+Access+(read-only)<http://lucene.apache.org/solr/version_control.html#Anonymous+Access+%28read-only%29>
> >>> >>
> >>> >> now you may need ant to build this.
> >>> >>
> >>> >> On Wed, May 20, 2009 at 11:53 AM, KK
> >>> >> wrote:
> >>> >> > I downloaded the nightly build . How do I get the trunk and where
> to
> >>> >> > get
> >>> >> > it
> >>> >> > from? I've never used trunks earlier . Please help me out. Tell me
> >>> >> > the
> >>> >> > detailed steps for all this.
> >>> >> >
> >>> >> > Thanks,
> >>> >> > KK.
> >>> >> >
> >>> >> > 2009/5/20 Noble Paul നോബിള് नोब्ळ्
> >>> >> >>
> >>> >> >> if you are using a trunk version you can still use the dataDir
> >>> >> >> attribute while
How to make wordDelimiterFilter[pulled from Solr nighly] to not break non-english words in a wrong way in lucene indexing/searching?
Hi All,
I'm trying to index some indian web page content which are basically a mix
of indian and say 5% of english content in the same page itself. For all
this I can not use standard or simple analyzer as they break the non-english
words in a wrong places say[because the isLetter(ch) happens to be false for
them, even if they are part of a word]. So I wrote/extended the anayzer that
does the following,
public class IndicAnalyzer extends Analyzer {
public TokenStream tokenStream(String fieldName, Reader reader) {
TokenStream ts = new WhitespaceTokenizer(reader);
//ts = new WordDelimiterFilter(ts, 1, 1, 1, 1, 0);
ts = new StopFilter(ts, StopAnalyzer.ENGLISH_STOP_WORDS);
ts = new LowerCaseFilter(ts);
ts = new PorterStemFilter(ts);
return ts;
}
}
This is working fine to some extent when the line commented above is left as
such, but its not able to give me the results when the documtnt contains a
string say "he...@how.com" and the searched query is hello, this is expected
as the above code doesnot do any word delimiting around these basic
characters [like @ . , etc ].
Now the problem is when I'm trying to use wordDelimiterFilter[hte commented
out line, this filter I got from solr] it is breaking say hindi words around
some characters which are actually part of a word. After going through the
code for this filter I found that it is using the isLetter() standard
fuction of java which I think returns false for those hindi characters
around which it is breaking the words. As per javadoc isLetter() is Unicode
compliant, right? so can't we say that it is aware of the above characters
that they are word delimiters, then why is this breaking around those
characters. I'm stuck and dont know how to get rid of the problem. And
because of this problem when I search for say a hindi word "helo" , assuming
its hindi, it highlights this word but alognwith that it also highlights the
letters of this word h/e/l/o whereever it finds it which it should not do,
right?
I request both Solr and Lucene users to guide me in fixing this issue. BTW,
do we need to do some sort of normalization for the content before sending
it to lucene indexer? just a thought, i don know whats the way out?
Re: How to support stemming and case folding for english content mixed with non-english content?
Note: I request Solr users to go through this mail and let me thier ideas.
Thanks Yonik, you rightly pointed it out. That clearly says that the way I'm
trying to mimic the default behaviour of Solr indexing/searching in Lucene
is wrong, right?.
I downloaded the latest version of solr nightly on may20[at that time I was
using Solr, now switched to Lucene]. I hope the issue must have been fixed
with that version.Anyway I'm going to download the latest nightly build
today and try it out. I hope using the nightly build instead of getting the
src from latest trunk is more or less same[provided I donwload the latest
nightly build, right?]as I don't know much about getting/compiling the src
from solr trunk. Do let me know if I've to use the trunk anyway, in that
case I'm ready to spend time to get that done.
BTW, Yonik, as per the basic Solr schema.xml file, the analyzers/filters
used by default are these ones, correct me if I'm wrong,
this is the code snip that mentions the filters used for indexing in Solr
and this is the part used for Solr querying,
To summarize the names are like this,
Indexing:
1. solr.WhitespaceTokenizerFactory -- tokenizer and the followings filters
as is clear from the name itself
2. solr.SynonymFilterFactory
3. solr.StopFilterFactory
4. solr.WordDelimiterFilterFactory (with the options as,
generateWordParts="1" generateNumberParts="1" catenateWords="1"
catenateNumbers="1" catenateAll="0" splitOnCaseChange="1")
5. solr.LowerCaseFilterFactory
6. solr.EnglishPorterFilterFactory
7. solr.RemoveDuplicatesTokenFilterFactory
Querying:
1. solr.WhitespaceTokenizerFactory
2. solr.SynonymFilterFactory
3. solr.StopFilterFactory
4. solr.WordDelimiterFilterFactory( options are: generateWordParts="1"
generateNumberParts="1" catenateWords="0" catenateNumbers="0"
catenateAll="0" splitOnCaseChange="1")
5. solr.LowerCaseFilterFactory
6. solr.EnglishPorterFilterFactory
7. solr.RemoveDuplicatesTokenFilterFactory
Now the filters/analyzers I used that tried to mimic the above behavior of
Solr [in Lucene] is as show below.
I pulled out the whitespacedelimiterfilter from Solr and my custom analyzer
for indexing is like this,
/**
* Analyzer for Indian language.
*/
public class IndicAnalyzerIndex extends Analyzer {
public TokenStream tokenStream(String fieldName, Reader reader) {
TokenStream ts = new WhitespaceTokenizer(reader);
ts = new WordDelimiterFilter(ts, 1, 1, 1, 1, 0); // I tried using
...(ts, 1, 1, 1, 1, 0, 1) 7 params, but no constructor found for that, I
didn't try to modify the code to add this feature though, then used this
with 6 params, that uses the constructor for which the last option for
splitOnCaseChange is set to 1 so we're doing the same thing even in this
way...
ts = new StopFilter(ts, StopAnalyzer.ENGLISH_STOP_WORDS);
ts = new LowerCaseFilter(ts);
ts = new PorterStemFilter(ts);
return ts;
}
}
and for querying this is teh code
/**
* Analyzer for Indian language.
*/
public class IndicAnalyzerQuery extends Analyzer {
public TokenStream tokenStream(String fieldName, Reader reader) {
TokenStream ts = new WhitespaceTokenizer(reader);
ts = new WordDelimiterFilter(ts, 1, 1, 0, 0, 0);
ts = new StopFilter(ts, StopAnalyzer.ENGLISH_STOP_WORDS);
ts = new LowerCaseFilter(ts);
ts = new PorterStemFilter(ts);
return ts;
}
}
The only difference for both is just the worddelimiterfilter with different
options... Comparing the analyzers/filters used by Solr and the above custom
analyzer we can see that I'm not using synonymfilter and
removeduplicatefilter. I hope these make sense for english content only and
using/skipping them will not make any differece to my non-english content.
Can someone with knowledge of Solr/Lulcene source code point me what exactly
is going wrong in my case whn I'm trying to do the same thing in Lucene. It
seems I'm missing some minor yet important thing...hence my custom
IndicAnalyzer is not behaving the way Solr's default anlyzer works and this
is clearly shown by Yonik that Solr is smart enough to detect unicoded word
endings and behaving as expected.
Any idea on this issue is welcome. Help me fix the issue. BTW, lucene ppl
when is that basic worddelimiterfilter going to be added to Lucene as well?
Any idea?
Thanks,
KK.
On Tue, Jun 9, 2009 at 7:01 PM, Yonik Seeley wrote:
> I just cut'n'pasted your word into Solr... it worked fine (it didn't
> split the word).
> Make sure you're using the latest from the trunk version of Solr...
> this was fixed since 1.3
&g
Configuring Apache SOLR with Multicore on IBM Websphere Application Server
Hi, I currently have Apache SOLR 3.6 running on Tomcat 7.0.27. I was able to successfully configure multicores too. This was my development environment and hence used tomcat - however the production environment is WAS. I need to migrate the existing multicores SOLR index from tomcat to WAS. Is there any documentation available on how to install SOLR on WebSphere and configure the multicores? Thanks, -Senthil
Embedded Server Issue : SOLRJ : No Such Core Found
Hi, I am facing an issue while trying to use the solrj EmbeddedServer to load a core. I am trying to load the example/solr packaged with solr-3.6.1. It works perfectly fine through CommonHTTPSolrSever. I am able to query and fetch the document. I used the following jar files to compile and run. apache-solr-solrj-3.6.1 apache-solr-core-3.6.1 slf4j-api-1.6.1 commons-io-2.1 lucene-core-3.6.1 commons-fileupload-1.2.2 servlet-api-2.5 commons-httpclient-3.1 commons-logging.jar commons-code-1.6 However if I try to load the same 'solr' through Embedded Server - i get No Such Core Error. 1) I tried passing the core name - 'collection1' - but same error >From solr.xml 2) I also checked my solrconfig.xml and requesthandler for "/update" is set as follows >From solrconfig.xml --Senthil
