RE: Solr over DRBD

Ypu should set a hearbeat and have the virtual IP setup for the active instance. So in haresources you can set like this: node1 IPaddr::10.2.0.11 drbddisk::r0 Filesystem::/dev/drbd0::/cluster/Solr::ext3::defaults,noatime httpd Are you running active/active cluster or active/passive? Francis

RE: cleanup old index directories on slaves

I use it in our env(Prod), it seems to working fine for years now only clean up the snapshot, but not the index. I added it to the cron that run once a day to clean up -francis -Original Message- From: Feak, Todd [mailto:todd.f...@smss.sony.com] Sent: Monday, October 05, 2009 2:34 PM T

RE: OutOfMemoryError due to auto-warming

I reduced the size of queryResultCache in solrconfig seems to fix the issue as well. 200 >From 500 500 Francis -Original Message- From: didier deshommes [mailto:dfdes...@gmail.com] Sent: Thursday, September 24, 2009 3:32 PM To: solr-user@lucene.apache.org Cc: Andrew Mont

RE: OutOfMemoryError due to auto-warming

You also can increase the JVM HeapSize if you have enough physical memory, like for example if you have 4GB physical, gives the JVM heapsize 2GB or 2.5GB. Francis -Original Message- From: didier deshommes [mailto:dfdes...@gmail.com] Sent: Thursday, September 24, 2009 3:32 PM To: solr-u

RE: OutOfMemory error on solr 1.3

Subject: Re: OutOfMemory error on solr 1.3 Just wondering, how much memory are you giving your JVM ? On Thu, Sep 10, 2009 at 7:46 AM, Francis Yakin wrote: > > I am having OutOfMemory error on our slaves server, I would like to know if > someone has the same issue and have the solution

RE: OutOfMemory error on solr 1.3

--Original Message- From: Constantijn Visinescu [mailto:baeli...@gmail.com] Sent: Wednesday, September 09, 2009 11:35 PM To: solr-user@lucene.apache.org Subject: Re: OutOfMemory error on solr 1.3 Just wondering, how much memory are you giving your JVM ? On Thu, Sep 10, 2009 at 7:46 AM, Fra

OutOfMemory issue after upgrade to 1.3 solr

Our slaves servers is having issue with the following error after we upgraded to Solr 1.3. Any suggestions? Thanks Francis NFO: [] webapp=/solr path=/select/ params={q=(type:artist+AND+alphaArtistSort:"forever+in+terror")} hits=1 status=0 QTime=1 SEVERE: java.lang.OutOfMemoryError: allocLar

RE: SolrJ and Solr web simultaneously?

ndencies... -Original Message- From: Francis Yakin [mailto:fya...@liquid.com] Sent: August-26-09 4:18 PM To: 'solr-user@lucene.apache.org' Subject: RE: SolrJ and Solr web simultaneously? We already opened port 80 from solr to DB so that's not the issue, but httpd(port 80) is very fla

RE: SolrJ and Solr web simultaneously?

y even try to implement triggers written in Java causing SOLR update on each row update (transactional); but I haven't heard anyone uses stored procs in Java, too risky and slow, with specific dependencies... -Original Message----- From: Francis Yakin [mailto:fya...@liquid.com] Sent: Augus

RE: SolrJ and Solr web simultaneously?

doing job with DB at the same time... and why to share I/O between SOLR and DB? Diversify, lower risks, having SOLR and DB on same box is extremely unsafe... -Fuad -Original Message- From: Francis Yakin [mailto:fya...@liquid.com] Sent: August-26-09 2:25 PM To: 'solr-user@lucene.apach

RE: SolrJ and Solr web simultaneously?

In addition, you will have nice Admin screen of standalone SOLR-master. -Fuad http://www.tokenizer.org -Original Message----- From: Francis Yakin [mailto:fya...@liquid.com] Sent: August-26-09 1:41 PM To: 'solr-user@lucene.apache.org'; Paul Tomblin Subject: RE: SolrJ and Solr web

RE: SolrJ and Solr web simultaneously?

I have the same situation now. If I don't want to use http connection, so I need to use EmbeddedSolrServer that what I think I need correct? We have Master/slaves solr, the applications use slaves for search. The Master only taking the new index from Database and slaves will pull the new index

RE: Example dir

and others http://wiki.apache.org/solr/SolrInstall#head-ec97d15a70656e9c0308009db70d71af3efc7cd2 2009/8/13 Francis Yakin > > Any one has any inputs for this? I really appreciated. > > Thanks > > Francis > > -Original Message----- > From: Francis Yakin [mailt

RE: Example dir

Any one has any inputs for this? I really appreciated. Thanks Francis -Original Message- From: Francis Yakin [mailto:fya...@liquid.com] Sent: Wednesday, August 12, 2009 3:39 PM To: 'solr-user@lucene.apache.org' Subject: Example dir As of right now when I installed and con

Example dir

As of right now when I installed and configure the Solr, I will get "example" dir ( like /opt/apache-solr-1.3.0/example ). How can I change that to something else, because "example" to me is not real? Thanks Francis

Solr Security

Have anyone had an experience to setup the Solr Security? http://wiki.apache.org/solr/SolrSecurity I would like to implement using HTTP Authentication or using Path Based Authentication. So, in the webdefault.xml I set like the following: Solr authenticated application /core

Synonyms.txt and index_synonyms.txt

Do you anyone the differences between these two? >From the schema.xml We have: Do you know if we need both of them for search to be

Segments_2 and segments.gen under Index folder and spellchecker1, spellchecker2, spellcheckerFile folder

I just upgraded our solr to 1.3.0 After I deployed the solr apps, I noticed there are: Segments_2 and segments.gen and there are 3 folder spellchecker1, spellchecker2 and spellcheckerFile What's these for? When I deleted them, I need bounce the apps again and it will generate the new ones aga

RE: Using curl comparing with using WebService::Solr

ache.org Subject: Re: Using curl comparing with using WebService::Solr On Fri, Jul 10, 2009 at 11:50 AM, Francis Yakin wrote: > I also commit too many I guess, since we have 1000 folders, so each loop > will executed the load and commit. > So 1000 loops with 1000 commits. I think it

RE: Using curl comparing with using WebService::Solr

ogram to post all these files in one request, you should be able to get better perf. the following can be the pseudo-code open connection write "" for each file write filecontent write "" close connection On Fri, Jul 10, 2009 at 10:23 AM, Francis Yakin wrote: > > I

RE: Using curl comparing with using WebService::Solr

I guess it opens as many http connections as there are files. if you can write a small program to post all these files in one request, you should be able to get better perf. the following can be the pseudo-code open connection write "" for each file write filecontent write "&

Using curl comparing with using WebService::Solr

I have about 1000 folders, each folder consist 2581 xml files. Total of xml files is ~ 2.6 millions I developed perl script, inside my script it's executed this cmd: curl http://localhost:7001/solr/update --data-binary "@0039000.xml" -H 'Content-type:text/plain; charset=utf-8' It tooks me ab

RE: Updating Solr index from XML files

Subject: Re: Updating Solr index from XML files If Perl is you choice: http://search.cpan.org/~bricas/WebService-Solr-0.07/lib/WebService/Solr.pm Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: Francis Yakin > To: "solr-user@lucene

Updating Solr index from XML files

I have the following "curl" cmd to update and doing commit to Solr ( I have 10 xml files just for testing) curl http://solr00:7001/solr/update --data-binary @xml_Artist-100170.txt -H 'Content-type:text/plain; charset=utf-8' curl http://solr00:7001/solr/update --data-binary @xml_Artist-101062.

RE: Is there any other way to load the index beside using "http" connection?

the commands from solr system to add and update this to the solr data/indexes. What commands do I have to use, for example the xml file named" /opt/test.xml" ? Thanks Francis -Original Message- From: Norberto Meijome [mailto:numard...@gmail.com] Sent: Sunday, July 05, 2

RE: Is there any other way to load the index beside using "http" connection?

apache-1.2.0/example/exampledocs/test.csv' -Yonik http://www.lucidimagination.com On Tue, Jul 7, 2009 at 1:37 PM, Francis Yakin wrote: > > I did try: > > curl > 'http://localhost:8983/solr/update/csv?stream.file=/opt/apache-1.2.0/example/exampledocs/test.csv"&am

RE: Is there any other way to load the index beside using "http" connection?

With curl 'http://localhost:8983/solr/update/csv?stream.file=/opt/apache-1.2.0/example/exampledocs/test.csv&stream.contentType=text/plain;charset=utf-8' No errors now. But , how can I verify if the update happening? Thanks Francis -Original Message- From: Francis Yak

RE: Is there any other way to load the index beside using "http" connection?

-Yonik http://www.lucidimagination.com On Mon, Jul 6, 2009 at 2:11 PM, Francis Yakin wrote: > > Ok, I have a CSV file(called it test.csv) from database. > > When I tried to upload this file to solr using this cmd, I got > "stream.contentType=text/plain: No such file or dire

Creating DataSource for DIH to Oracle Database

Have any one had experience creating a datasource for DIH to an Oracle Database? Also, from the Solr side we are running weblogic and deploy the application using weblogic. I know in weblogic we can create a datasource that can connect to Oracle database, has any one had experience with this?

RE: Is there any other way to load the index beside using "http" connection?

r instance. Regards Nitin Francis Yakin wrote: > > > Ok, I have a CSV file(called it test.csv) from database. > > When I tried to upload this file to solr using this cmd, I got > "stream.contentType=text/plain: No such file or directory" error > > curl > http://

RE: Is there any other way to load the index beside using "http" connection?

s/test.csv&stream.contentType=text/plain;charset=utf-8 -bash: stream.contentType=text/plain: No such file or directory undefined field cat What did I do wrong? Francis -Original Message- From: Norberto Meijome [mailto:numard...@gmail.com] Sent: Monday, July 06, 2009 11:01 AM To: Francis

RE: Is there any other way to load the index beside using "http" connection?

uly 06, 2009 4:33 AM To: Francis Yakin Cc: solr-user@lucene.apache.org Subject: Re: Is there any other way to load the index beside using "http" connection? On Sun, 5 Jul 2009 10:28:16 -0700 Francis Yakin wrote: [...]> > >upload the file to your SOLR server? Then the data f

RE: Is there any other way to load the index beside using "http" connection?

# Warning: Note the clean command which will wipe your index... GET "http:// $server:$port/$path/dataimport?command=full-import&clean=true&optimize=true" Hope this helps out some. Cheers //Marcus On Sun, Jul 5, 2009 at 7:28 PM, Francis Yakin wrote: > Norberto, >

RE: Is there any other way to load the index beside using "http" connection?

ues that are rooted in the fact that you are posting via >HTTP (vs embedded solr or DIH) aren't going to go away. But it's the simpler >approach without changing too much of your current setup. -Original Message- From: Norberto Meijome [mailto:numard...@gmail.com] Sent: Sunda

RE: Is there any other way to load the index beside using "http" connection?

ned by other posters - you need to make sure the versions of Lucene and Solr are compatible (use same jars), you use the same Analyzers, and you create the appropriate 'schema' that Solr understands. -glen 2009/7/2 Francis Yakin : > > Glen, > > Database we use is Oracle, I am

RE: Is there any other way to load the index beside using "http" connection?

Sent: Thursday, July 02, 2009 3:01 AM To: solr-user@lucene.apache.org Cc: Francis Yakin Subject: Re: Is there any other way to load the index beside using "http" connection? On Wed, 1 Jul 2009 15:07:12 -0700 Francis Yakin wrote: > > We have several thousands of xml files in d

RE: Is there any other way to load the index beside using "http" connection?

: Is there any other way to load the index beside using "http" connection? did you explore DIH http://wiki.apache.org/solr/DataImportHandler it has features to import from Db, xml files etc On Thu, Jul 2, 2009 at 3:37 AM, Francis Yakin wrote: > > We have several thousands

RE: Is there any other way to load the index beside using "http" connection?

ou can use EmbeddedSolrServer. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message ---- > From: Francis Yakin > To: "solr-user@lucene.apache.org" > Sent: Wednesday, July 1, 2009 6:07:12 PM > Subject: Is there any other way to l

RE: Is there any other way to load the index beside using "http" connection?

.php/LuSql 2009/7/1 Francis Yakin : > > We have several thousands of xml files in database that we load it to solr > master > The Database uses "http" connection and transfer those files to solr master. > Solr then translate xml files to their lindex. > > We are expe

RE: Is there any other way to load the index beside using "http" connection?

e using LuSql[1]. It is faster than Solr, sometimes as much as an order of magnitude faster. Disclosure: I am the author of LuSql -Glen http://zzzoot.blogspot.com/ [1]http://lab.cisti-icist.nrc-cnrc.gc.ca/cistilabswiki/index.php/LuSql 2009/7/1 Francis Yakin : > > We have several thous

RE: Is there any other way to load the index beside using "http" connection?

can also import documents as csv data/file. Finally, you can use EmbeddedSolrServer. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: Francis Yakin > To: "solr-user@lucene.apache.org" > Sent: Wednesday, July 1, 2009 6:07:1

Is there any other way to load the index beside using "http" connection?

We have several thousands of xml files in database that we load it to solr master The Database uses "http" connection and transfer those files to solr master. Solr then translate xml files to their lindex. We are experiencing issue with close/open connection in the firewall and very very sl

RE: Slowness during submit the index

cis, >> >> I can't tell what the problem is from the information you've >> provided so far. My gut instinct is that this is due to some >> difference in QA vs. PROD environments that isn't Solr-specific. >> >> Otis >> -- >> Sematext --

RE: Slowness during submit the index

t of data, different network speed, different hardware...) Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message ---- > From: Francis Yakin > To: "solr-user@lucene.apache.org" > Sent: Friday, June 19, 2009 10:39:48 PM > Subject: RE: Slo

RE: Slowness during submit the index

on to the DB the same on both machines * are both the PROD and QA DB servers the same and are both DB instances the same ... Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: Francis Yakin > To: "solr-user@lucene.apache.org"

RE: Java OutOfmemory error during autowarming

[mailto:hossman_luc...@fucit.org] Sent: Friday, June 19, 2009 5:49 PM To: 'solr-user@lucene.apache.org' Subject: RE: Java OutOfmemory error during autowarming : Date: Mon, 1 Jun 2009 11:34:08 -0700 : From: Francis Yakin : Subject: RE: Java OutOfmemory error during autowarming ... :

Slowness during submit the index

We are experiencing slowness during reloading/resubmitting index from Database to the master. We have two environments: QA and Prod. The slowness is happened only in Production but not in QA. It only takes one hours to reload 2.5Mil indexes compare 5-6 hours to load the same size of index in

Can I use the same index from 1.2.0 to 1.3.0?

Can I transport the index from Solr 1.2 to Sol 1.3 without resubmiting/reloading again from Database? Francis

RE: OutOfMemory error on solrslaves

17, 2009 8:28 PM To: solr-user@lucene.apache.org Subject: Re: OutOfMemory error on solrslaves Francis Yakin wrote: > We are experiencing "OutOfMemory" error frequently on our slaves, this is the > error: > > SEVERE: Error during auto-warming of > key:org.apache.

OutOfMemory error on solrslaves

We are experiencing "OutOfMemory" error frequently on our slaves, this is the error: SEVERE: Error during auto-warming of key:org.apache.solr.search.queryresult...@a8c6f867:java.lang.OutOfMemoryError: allocLargeObjectOrArray - Object size: 5120080, Num elements: 1280015 java.lang.OutOfMemoryEr

RE: Upgrading 1.2.0 to 1.3.0 solr

ed soon. > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > >> From: Francis Yakin >> To: "solr-user@lucene.apache.org" >> Sent: Wednesday, June 10, 2009 1:17:25 AM >> Subject: Upgrading 1.2.0 to 1.3.0 solr >>

Example folder - can we change it?

When I install solr , by default it will install it under /opt/apache-solr-1.3.0/ The bin , config file and data is under /opt/apache-solr-1.3.0/example/solr Is there anyway that we change the example to something else? Because "example" is can be interpreted wrong ( like sample, so it's not re

RE: Solr.war

build.xml: Koji Francis Yakin wrote: > We are planning to upgrade solr 1.2.0 to 1.3.0 > > Under 1.3.0 - Which of war file that I need to use and deploy on my > application? > > We are using weblogic. > > There are two war files under > /opt//apache-solr-1

Solr.war

We are planning to upgrade solr 1.2.0 to 1.3.0 Under 1.3.0 - Which of war file that I need to use and deploy on my application? We are using weblogic. There are two war files under /opt//apache-solr-1.3.0/dist/apache-solr-1.3.0.war and under /opt/apache-solr-1.3.0/example/webapps/solr.war. Wh

RE: Java OutOfmemory error during autowarming

he.org/solr/SolrCaching) 2009/5/29 Francis Yakin > > I know, but the FieldCache is not in the solrconfig.xml > > > -Original Message- > From: Yonik Seeley [mailto:ysee...@gmail.com] > Sent: Friday, May 29, 2009 10:47 AM > To: solr-user@lucene.apache.org > Subjec

RE: Java OutOfmemory error during autowarming

I know, but the FieldCache is not in the solrconfig.xml -Original Message- From: Yonik Seeley [mailto:ysee...@gmail.com] Sent: Friday, May 29, 2009 10:47 AM To: solr-user@lucene.apache.org Subject: Re: Java OutOfmemory error during autowarming On Fri, May 29, 2009 at 1:44 PM, Francis

RE: Java OutOfmemory error during autowarming

decrease the memory consumption. -Yonik http://www.lucidimagination.com On Fri, May 29, 2009 at 1:02 AM, Francis Yakin wrote: > During auto-warming of solr search on QueryResultKey, Our Production > solrslaves errors throw OutOfMemory error and application need to be bounced. > > He

Java OutOfmemory error during autowarming

During auto-warming of solr search on QueryResultKey, Our Production solrslaves errors throw OutOfMemory error and application need to be bounced. Here is the error logs: SEVERE: Error during auto-warming of key:org.apache.solr.search.queryresult...@9acd5d67

RE: Solrconfig.xml

e.org Subject: Re: Solrconfig.xml Is there an error in the logs? On May 6, 2009, at 2:12 PM, Francis Yakin wrote: > > I just upgraded from 1.2.0 to 1.3.0 of solr. > We have an existing data/index that I will be using from 1.2.0 to > 1.3.0 and I use the default solrconfig.xml that come fr

Solrconfig.xml

I just upgraded from 1.2.0 to 1.3.0 of solr. We have an existing data/index that I will be using from 1.2.0 to 1.3.0 and I use the default solrconfig.xml that come from 1.3.0. For some reason when I used solrconfig.xml from 1.2.0 it works and I can see the index and data, but I used solrconfig.

RE: OutOfMemory error

iginal Message > From: Francis Yakin > To: "solr-user@lucene.apache.org" > Sent: Tuesday, May 5, 2009 1:00:07 PM > Subject: OutOfMemory error > > > I am having frequent OutOfMemory error on our slaves server. > > SEVERE: Error during auto-

Upgrading from 1.2.0 to 1.3.0

What's the best way to upgrade solr from 1.2.0 to 1.3.0 ? We have the current index that our users search running on 1.2.0 Solr version. We would like to upgrade it to 1.3.0? We have Master/Slaves env. What's the best way to upgrade it without affecting the search? Do we need to do it on mast

OutOfMemory error

I am having frequent OutOfMemory error on our slaves server. SEVERE: Error during auto-warming of key:org.apache.solr.search.queryresult...@aca6b9cb:java.lang.OutOfMemoryError: allocLargeObjectOrArray - Object size: 34279632, Num elements: 8569904 SEVERE: Error during auto-warming of key:org.a