Re: solr different sizes on master and slave

2011-03-01 Thread Markus Jelsma
, Mar 1, 2011 at 3:26 PM, Mike Franon wrote: > >> No pending commits, what it looks like is there are almost two copies > >> of the index on the master, not sure how that happened. > >> > >> > >> > >> On Tue, Mar 1, 2011 at 3:08 PM, Markus Jelsma > >> &g

Re: Indexed, but cannot search

2011-03-01 Thread Markus Jelsma
which is how I was able to get the regular search working. I have not > however been able to get the search across all fields to work. > > On Tue, Mar 1, 2011 at 3:01 PM, Markus Jelsma wrote: > > Traditionally, people forget to reindex ;) > > > > > Hi all, > &g

Re: Split analysis

2011-03-02 Thread Markus Jelsma
e easily, but how can I get access to the > original (now tokenized and filtered) input > > Many thanks in advance, > Dan -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350

Re: solr different sizes on master and slave

2011-03-02 Thread Markus Jelsma
replication.properties. On Wednesday 02 March 2011 15:03:54 Mike Franon wrote: > Is it ok if I just delete the old copies manually? or maybe run a > script that does it? > > On Tue, Mar 1, 2011 at 7:47 PM, Markus Jelsma > > wrote: > > Indeed, the slave should not have us

Re: Indexed, but cannot search

2011-03-02 Thread Markus Jelsma
gt; - > > 1.0 > - > > 1.0 > - > > 1.0 > > - > > 0.0 > > - > > 0.0 > > - > > 0.0 > > - > > 0.0 > > - > > 0.0 > > > - > > 0.0 > - > > 0.0 > > - > > 0.0 > >

Re: MLT with boost

2011-03-02 Thread Markus Jelsma
by > > >> MLT? If not out of the box > >> how would one go about achieving this functionality? > >> > >> Thanks > > > > Beside the point, why do you need such function? > > If you give us more information/background of your needs, it

Re: Formatting the XML returned

2011-03-02 Thread Markus Jelsma
If you're confortable with XSL you can create a transformer and use Solr's XSLTResponseWriter to do the job. http://wiki.apache.org/solr/XsltResponseWriter > Hi all, > > This list has proven itself quite useful since I got started with Solr. I'm > wondering if it is possible to dictate the XML t

Re: memory leak during undeploying

2011-03-02 Thread Markus Jelsma
Hi, I remember reading somewhere that undeploying an application in Tomcat won't release memory, thus repeating the cycle will indeed exhaust the permgen. You could enable garbage collection of the permgen. HotSpot can do this for you but it depends on using CMS which you might not want to us

Re: Solr Admin Interface, reworked - Go on? Go away?

2011-03-02 Thread Markus Jelsma
Nice! It makes multi core navigation a lot easier. What license do the icons have? > Hi List, > > given that fact that my java-knowledge is sort of non-existing .. my > idea was to rework the Solr Admin Interface. > > Compared to CouchDBs Futon or the MongoDB Admin-Utils .. not that fancy, > bu

Re: perfect match in dismax search

2011-03-03 Thread Markus Jelsma
Use either the string fieldType or a field with very little analysis (KeywordTokenizer + LowercaseFilter). > How to obtain perfect match with dismax query?? > > es: > > i want to search "hello i love you" with deftype=dismax in the title field > and i want to obtain results which title is exact

Re: Selection Between Solr and Relational Database

2011-03-03 Thread Markus Jelsma
Well, an RDBMS can be very fast but Solr using fq can be very fast as well. Just try fq=group:sports&fq=createdtime: > Dear all, > > I have started to learn Solr for two months. At least right now, my system > runs good in a Solr cluster. > > I have a question when implementing one feature in m

Re: Solr TermsComponent: space in term

2011-03-03 Thread Markus Jelsma
er mailing list archive at > Nabble.com. -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350

Re: adding a document using curl

2011-03-03 Thread Markus Jelsma
ment to the example Solr instance. > > I have tried a few things however they seem to be for the file on the same > server as solr, in my case I am pushing the document from a windows > machine to Solr for indexing. > > Ta > Ken -- Markus Jelsma - CTO - Openindex http://

Re: Error during auto-warming of key

2011-03-08 Thread Markus Jelsma
Anyone here with some thoughts on this issue? > Hi, > > Yesterday's error log contains something peculiar: > > ERROR [solr.search.SolrCache] - [pool-29-thread-1] - : Error during auto- > warming of key:+*:* > (1.0/(7.71E-8*float(ms(const(1298682616680),date(sort_date)))+1.0))^20.0:ja > va.lang.

Re: two QueryHandler components in one schema?

2011-03-08 Thread Markus Jelsma
A request handler can have first-components and last-components and also just plain components. List all your stuff in components and voila. Don't forget to also add debug, facet and other default components if you need them. > Le 8 mars 2011 à 23:03, Chris Hostetter a écrit : > > : in my schema

Re: Solr UIMA Wiki page

2011-03-09 Thread Markus Jelsma
Great work! On Wednesday 09 March 2011 11:20:41 Tommaso Teofili wrote: > Hi all, > I just improved the Solr UIMA integration wiki page [1] so if anyone is > using it and/or has any feedback it'd be more than welcome. > Regards, > Tommaso > > [1] : http://wiki.apache.org

Re: NRT in Solr

2011-03-09 Thread Markus Jelsma
uments other Cores < 100.000 > > - Solr1 for Search-Requests - commit every Minute - 5GB Xmx > - Solr2 for Update-Request - delta every Minute - 4GB Xmx > -- > View this message in context: > http://lucene.472066.n3.nabble.com/NRT-in-Solr-tp2652689p2654696.html Sent &

Re: True master-master fail-over without data gaps

2011-03-09 Thread Markus Jelsma
RAMdisk > ...but the index resides on disk doesn't it??? lol > > -Original Message- > From: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com] > Sent: Wednesday, March 09, 2011 9:06 AM > To: solr-user@lucene.apache.org > Subject: Re: True master-master fail-over without data gaps > >

FunctionQueries and FieldCache and OOM

2011-03-09 Thread Markus Jelsma
Hi, In one of the environments i'm working on (4 Solr 1.4.1. nodes with replication, 3+ million docs, ~5.5GB index size, high commit rate (~1-2min), high query rate (~50q/s), high number of updates (~1000docs/commit)) the nodes continuously run out of memory. During development we frequently r

Re: Possible to sort in .xml file?

2011-03-10 Thread Markus Jelsma
sn't seem to offer the option to pass in this value > :( > > TIA -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350

Re: Possible to sort in .xml file?

2011-03-10 Thread Markus Jelsma
Hi, > > Thanks for the quick reply! > > I did a quick look in the solrconfig.xml file, but can't see anything about > "sort", appart from: > > > > > TIA > > Andy > > On Thu, Mar 10, 2011 at 10:33 AM, Markus Jelsma > > wro

Re: Math-generated fields during query

2011-03-10 Thread Markus Jelsma
>q=boxname:findbox+_val_:"product(product(x,y),z)"&fl=*,score > ...where score will contain the resultant volume. > > Is there a trick to getting not a score, but the actual value of > quantity*price (e.g. product(5,2.21) == 11.05)? > > Many thanks -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350

Re: FunctionQueries and FieldCache and OOM

2011-03-10 Thread Markus Jelsma
ut without continues updates and without stress tests. Firing manual queries with different values for the bf parameter don't show any difference in the values listed on the stats page. Someone cares to provide an explanation? Thanks On Wednesday 09 March 2011 22:21:19 Markus Jelsma wrote: &g

Re: Math-generated fields during query

2011-03-10 Thread Markus Jelsma
IRA)? > > Thanks again! > Peter > > > On Thu, Mar 10, 2011 at 1:19 PM, Markus Jelsma > > wrote: > > Not at the moment if i'm not mistaken. The same issue is with Solr 3.1 > > where relative distances are not being returned as field value when > > doin

Re: FunctionQueries and FieldCache and OOM

2011-03-10 Thread Markus Jelsma
from excessive memory consumption: recip(ms(NOW/,),,1,1) On Thursday 10 March 2011 15:14:25 Markus Jelsma wrote: > Well, it's quite hard to debug because the values listed on the stats page > in the fieldCache section don't make much sense. Reducing precision with > NOW/HOUR,

Re: Question about Term Vectors

2011-03-14 Thread Markus Jelsma
You need to reindex. On Monday 14 March 2011 14:04:00 Ahsan |qbal wrote: > Hi All > > Is there any way to drop term vectors from already built index file. > > Regards > Ahsan Iqbal -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350

Re: Solr 1.4 replication - partial index on slave while indexing master

2011-03-14 Thread Markus Jelsma
- > Should I disable replication while indexing master or should i use > core swaping? > > Thanks. -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350

Re: Solr 1.4 replication - partial index on slave while indexing master

2011-03-14 Thread Markus Jelsma
In solrconfig there might be a autocommit section enabled. On Monday 14 March 2011 14:18:42 lame wrote: > I don't commit at all we use Dataimporter, but I have a feeling that > it could be done by DIH (autocommit is it possible)? > > 2011/3/14 Markus Jelsma : > > Do you

Re: Results driving me nuts!

2011-03-14 Thread Markus Jelsma
> than this, including some things I wouldn't want, or if there is some > other parameter that would do this instead? > > Thanks for any advice, > > Jonathan -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350

Re: Solr 1.4 replication - partial index on slave while indexing master

2011-03-14 Thread Markus Jelsma
10 > 2147483647 > 1 > false > > > > 10 > > > But as you said before slave replicates after commit, but in that case > shouldn't master be also updated with nex index? Our scripts showed > that master still has the old ind

Re: Dynamically boost search scores

2011-03-14 Thread Markus Jelsma
/solr/search?q=dog&boost=true > > The score of each document would be boosted by the number in the field > boost_score. > > Unfortunately, I have no idea how to implement this actually but I'm hoping > that's where you all can come in. > > Thanks, > >

WDF, automatic phrase queries and omitTermFreqAndPositions

2011-03-14 Thread Markus Jelsma
;s not possible yet. Any advice? Cheers, -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350

Re: WDF, automatic phrase queries and omitTermFreqAndPositions

2011-03-14 Thread Markus Jelsma
https://issues.apache.org/jira/browse/SOLR-2015 On Monday 14 March 2011 16:47:24 Markus Jelsma wrote: > Hi, > > In Solr 1.4.1 we don't have feature to disable automatic generation of > phrase queries. The phrase queries are generated thanks of the word > delimiter filter i use. The pro

Re: Results driving me nuts!

2011-03-14 Thread Markus Jelsma
t right? If so, that's exactly what I want to try > out, excellent. > > On 3/14/2011 10:48 AM, Markus Jelsma wrote: > > You can use omitNorms="true" for any given field. Length normalization > > will be disabled and index-time boosting will not be available any more

Re: Solr 1.4 replication - partial index on slave while indexing master

2011-03-14 Thread Markus Jelsma
; that is the case. > If you don't have any other ideas I'll probably try reindexing second > core, than swap cores and run delta import (to import documets added > in the meantime). > > Thanks > > 2011/3/14 Markus Jelsma : > > These settings don't a

Re: Solr performance issue

2011-03-14 Thread Markus Jelsma
Hi Doğacan, Are you, at some point, running out of heap space? In my experience, that's the common cause of increased load and excessivly high response times (or time outs). Cheers, > Hello everyone, > > First of all here is our Solr setup: > > - Solr nightly build 986158 > - Running solr in

Re: Solr performance issue

2011-03-14 Thread Markus Jelsma
> Hello, > > 2011/3/14 Markus Jelsma > > > Hi Doğacan, > > > > Are you, at some point, running out of heap space? In my experience, > > that's the common cause of increased load and excessivly high response > > times (or time > > outs). &g

Re: Solr performance issue

2011-03-14 Thread Markus Jelsma
> Nope, no OOM errors. That's a good start! > Insanity count is 0 and fieldCAche has 12 entries. We do use some boosting > functions. > > Btw, I am monitoring output via jconsole with 8gb of ram and it still goes > to 8gb every 20 seconds or so, > gc runs, falls down to 1gb. Hmm, maybe the garb

Re: Solr performance issue

2011-03-14 Thread Markus Jelsma
You might also want to add the following switches for your GC log. > JAVA_OPTS="$JAVA_OPTS -verbose:gc -XX:+PrintGCTimeStamps > -XX:+PrintGCDetails - Xloggc:/var/log/tomcat6/gc.log" -XX:+PrintGCApplicationConcurrentTime -XX:+PrintGCApplicationStoppedTime > > Also, what JVM version are you using

Re: Solr performance issue

2011-03-14 Thread Markus Jelsma
so often. I have had good luck with my Solr > using this: -XX:+UseParallelGC > > On 3/14/2011 4:15 PM, Doğacan Güney wrote: > > Hello again, > > > > 2011/3/14 Markus Jelsma > > > >>> Hello, > >>> > >>> 2011/3/14 Markus Jelsma > >

Re: Solr performance issue

2011-03-14 Thread Markus Jelsma
> Hello, > > 2011/3/14 Markus Jelsma > > > That depends on your GC settings and generation sizes. And, instead of > > UseParallelGC you'd better use UseParNewGC in combination with CMS. > > JConsole now shows a different profile output but load is still high and

Re: Solr performance issue

2011-03-15 Thread Markus Jelsma
onment: > > -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode > > I've never had a real problem with memory, so I've not done any kind of > auditing. I probably should, but time is a limited resource. > > Shawn > > On 3/14/2011 2:29 PM, Markus Jelsma wrote: >

Re: Solr admin page timed out and index updating issues

2011-03-16 Thread Markus Jelsma
ccess > the solr admin page? > > Please...! Thanks! > > - > loredanaebook.it > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-admin-page-timed-out-and-index-upd > ating-issues-tp2664429p2676437.html Sent from the Solr - User mailing l

Re: Stemming question

2011-03-16 Thread Markus Jelsma
> stemmed and replace it with the root work in the index. > I verified this by looking at analysis.jsp. > > Is there an option to expand the stemmer to include all combinations of the > word? Like include 's, ly, etc? > > Other options besides protection? > > B

Re: Multicore

2011-03-16 Thread Markus Jelsma
l file, I get those errors. > > Now this says to me I am not loading a library or two somewhere but I've > looked through the configuration files and cannot see any other place other > than solrconfig.xml where that would be set so what am I doing incorrectly? > > Thanks, > > Brian Lamb -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350

Re: Error during auto-warming of key

2011-03-16 Thread Markus Jelsma
> that is odd... > > can you let us know exactly what verison of Solr/Lucne you are using (if > it's not an official release, can you let us know exactly what the version > details on the admin info page say, i'm curious about the svn revision) Of course, that's the stable 1.4.1. > > can you al

Re: FunctionQueries and FieldCache and OOM

2011-03-16 Thread Markus Jelsma
Hi, > FWIW: it sounds like your problem wasn't actually related to your > fieldCache, but probably instead if was because of how big your > queryResultCache is It's the same cluster as in the other thread. I decided a long time ago that documentCache and queryResultCache wouldn't be a good

Re: Error during auto-warming of key

2011-03-16 Thread Markus Jelsma
Actually, i dug in the logs again and surprise, it sometimes still occurs with `random` queries. Here's are a few snippets from the error log. Somewhere during that time there might be OOM-errors but older logs are unfortunately rotated away. 2011-03-14 00:25:32,152 ERROR [solr.search.SolrCac

Re: memory not getting released in tomcat after pushing large documents

2011-03-17 Thread Markus Jelsma
l and privileged material > for the sole use of the intended recipient. Any unauthorized review, > use or distribution by others is strictly prohibited. If you have > received the message in error, please advise the sender by reply > email and delete the message. Thank you." > -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350

Re: Info about Debugging SOLR in Eclipse

2011-03-17 Thread Markus Jelsma
gt; for the sole use of the intended recipient. Any unauthorized review, > use or distribution by others is strictly prohibited. If you have > received the message in error, please advise the sender by reply > email and delete the message. Thank you." > **

Re: dismax 1.4.1 and pure negative queries

2011-03-17 Thread Markus Jelsma
Hi, It works just as expected, but not in a phrase query. Get rid of your quotes and you'll be fine. Cheers, > Should 1.4.1 dismax query parser be able to handle pure negative queries > like: > > &q="-foo" > &q="-foo -bar" > > It kind of seems to me trying it out that it can NOT. Can anyone

Re: dismax 1.4.1 and pure negative queries

2011-03-17 Thread Markus Jelsma
* TO *]. > My fault for putting in the quotes in the email, I actually don't have > tests in my quotes, just tried again to make sure. > > And I always get 0 results on a pure negative Solr 1.4.1 dismax query. I > think it does not actually work? > > On 3/17/2011 3:52 PM,

Re: How to get stopwords and synonyms files for several lanuages

2011-03-18 Thread Markus Jelsma
directory. > > Thanx, > Alex > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/How-to-get-stopwords-and-synonyms-files > -for-several-lanuages-tp2698494p2698494.html Sent from the Solr - User > mailing list archive at Nabble.com. -- Markus

Re: How to get stopwords and synonyms files for several lanuages

2011-03-18 Thread Markus Jelsma
ttp://lucene.472066.n3.nabble.com/How-to-get-stopwords-and-synonyms-files > -for-several-lanuages-tp2698494p2698593.html Sent from the Solr - User > mailing list archive at Nabble.com. -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350

Re: Snappull failed

2011-03-19 Thread Markus Jelsma
Anyone? > Hi, > > There are a couple of Solr 1.4.1 slaves, all doing the same. Pulling some > snaps, handling some queries, nothing exciting. But can anyone explain a > sudden nightly occurence of this error? > > 2011-02-16 01:23:04,527 ERROR [solr.handler.ReplicationHandler] - > [pool-238- thre

Re: Problems with creating a query that matches all the documents I want to display

2011-03-21 Thread Markus Jelsma
Hi, In a _high load_ environment with queryResultCache and filterCache enabled, you need to be very careful about using NOW in its highest precision. It adds - and likely evicts - a non retrievable entry in both caches, wasting a lot of resources. It's best to always reduce precision when you

Re: Search failing for matched text in large field

2011-03-23 Thread Markus Jelsma
Enable TermVectors for fields that you're going tot highlight. If it is disabled Solr will reanalyze the field, killing performance. > I looked into the search that I'm doing a little closer and it seems > like the highlighting is slowing it down. If I do the query without > requesting highlighti

Re: Multiple Cores with Solr Cell for indexing documents

2011-03-24 Thread Markus Jelsma
der.java:307) > at java.lang.ClassLoader.loadClass(ClassLoader.java:248) > ... 27 more > > RequestURI=/solr/core0/update/extract href="http://jetty.mortbay.org/";>Powered by > Jetty:// > > > > > > > > > > > > > > > > > > > > > > > Any assistance you could provide or installation guides/tutorials/etc. that > you could link me to would be greatly appreciated. Thank you all for your > time! > > ~Brandon Waterloo -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350

Re: Multiple Cores with Solr Cell for indexing documents

2011-03-24 Thread Markus Jelsma
different library modules for each. > > The relevant line in my solr.xml file is sharedLib="lib">. That is housed in .../example/solr/. So, does it look > in .../example/lib or .../example/solr/lib? > > ~Brandon Waterloo > ___

Re: Multiple Cores with Solr Cell for indexing documents

2011-03-25 Thread Markus Jelsma
You can only set properties for a lib dir that must be used in solrconfig.xml. You can use sharedLib in solr.xml though. > There's options in solr.xml that point to lib dirs. Make sure you get > them right. > > Upayavira > > On Thu, 24 Mar 2011 23:28 +0100, "Marku

Re: problems indexing web content

2011-03-28 Thread Markus Jelsma
The analyzer order doesn't really matter, char filters are regardless of position in the analyzer always executed first. Multiple filters of the same type, however, are affected by order. Also, your error is not caused by a faulty analyzer, there is something wrong in your XML. Anyway, accordi

Re: problems indexing web content

2011-03-28 Thread Markus Jelsma
Also, don't forget to encode entities or wrap them in CDATA. > Jan, > > thank you for such a quick reply. I have a feed coming in that I convert to > an Here is the type for text including index > and query with the changes suggested. > > > positionIncrementGap="100"> >

Re: problems indexing web content

2011-03-28 Thread Markus Jelsma
[1]: http://lucene.apache.org/solr/api/org/apache/solr/schema/BinaryField.html > > On Mar 28, 2011, at 1:38 PM, Markus Jelsma wrote: > > Also, don't forget to encode entities or wrap them in CDATA. > > > >> Jan, > >> > >> thank you for such a q

Re: copyField at search time / multi-language support

2011-03-28 Thread Markus Jelsma
https://issues.apache.org/jira/browse/SOLR-1979 > Tom, > > Could you share the method you use to perform language detection? Any open > source tools that do that? > > Thanks. > > --- On Mon, 3/28/11, Tom Mortimer wrote: > > From: Tom Mortimer > > Subject: copyField at search time / multi-lang

Re: copyField at search time / multi-language support

2011-03-28 Thread Markus Jelsma
I haven't tried this as an UpdateProcessor but it relies on Tika and that LanguageIdentifier works well, except for short texts. > Thanks Markus. > > Do you know if this patch is good enough for production use? Thanks. > > Andy > > --- On Tue, 3/29/11, Markus Jelsm

Re: how to start GarbageCollector

2011-03-29 Thread Markus Jelsma
I seriously doubt heap usage is actually your problem. Usually a garbage collector is running, if it (somehow) doesn't you will definately run out of memory some time. Where did you check memory usage? > Hello, > > my problem is, that after a full-import solr reserved all of my RAM and my >

Re: how to start GarbageCollector

2011-03-29 Thread Markus Jelsma
Unix tools won't show heap usage statistics. Please use tools that come with your JVM such as jps, jtop, jstat or setup monitoring over JMX to get a good picture. All aside, RAM is most likely not your problem. > i run an full-import via DIH, 35 Million Documents, i dont restart solr. my > cron

Re: 2 index within the same Solr server ?

2011-03-29 Thread Markus Jelsma
http://wiki.apache.org/solr/CoreAdmin > Hello every body, > > Is it possible to create 2 index within the same Solr server ? > > Thank you. > > Amel.

Re: Concatenate multivalued DIH fields

2011-03-29 Thread Markus Jelsma
Haven't tried your use case but i believe DIH's ScriptTransformer can do the trick. It seems to operate on rows so you can fetch both fields and add a concatenated field. http://wiki.apache.org/solr/DataImportHandler#ScriptTransformer > I have two multivalued DIH fields "fname" and "lname". I w

Re: Javabin->JSon

2011-03-29 Thread Markus Jelsma
You've asked this twice now. This is a Java specific question and unless someone feels like answering i'd try googling somewhere else. > Hi guys, > > I have a Javabin object and I need to convert that to a JSon object. How ? > pls help? > I am using solrj (client) that doesn't support JSON so (

Re: Matching on a multi valued field

2011-03-29 Thread Markus Jelsma
Hi, Your filter query is looking for a match of "man's friend" in a single field. Regardless of analysis of the common_names field, all terms are present in the common_names field of both documents. A multiValued field is actually a single field with all data separated with positionIncrement. T

Re: Matching on a multi valued field

2011-03-29 Thread Markus Jelsma
orly, all replies came in while sending =) > Hi, > > Your filter query is looking for a match of "man's friend" in a single > field. Regardless of analysis of the common_names field, all terms are > present in the common_names field of both documents. A multiValued field > is actually a single fi

Re: Challenges of bundling Solr out-of-box

2011-03-29 Thread Markus Jelsma
Hi, You're right, there are new technical challenges for customers that don't have the experience in-house. Some customers have personnel you can teach how to monitor and maintain an installation. Others just take a service level agreement or just let it run forever without issues, if the envir

Re: how to start GarbageCollector

2011-03-30 Thread Markus Jelsma
Stockii, don't be sad but read about JVM memory usage and collection. Increasing and decreasing memory consumption is normal, you would only worry if the left bar reaches 100% and stays at 100%. Start overhere: http://download.oracle.com/javase/6/docs/technotes/guides/management/jconsole.html B

Re: overwriting entries when indexing

2011-03-30 Thread Markus Jelsma
hat doesn't seem to be the case. Is there anything I can > do to force overwriting? > The way we index it now is (to put it simply): > CommonsHttpSolrServer.add(List) > > Any help appreciated, thanks! > > - Carl-Erik -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350

Re: overwriting entries when indexing

2011-03-30 Thread Markus Jelsma
Are you actually sending in documents with the field specified in uniqueKey with existing values? On Wednesday 30 March 2011 13:59:15 Carl-Erik Herheim wrote: > Yes, I have. > > Den 30.03.2011 13:41, skrev Markus Jelsma: > > Have you defined a uniqueKey in your schema

Re: Special characters index

2011-03-30 Thread Markus Jelsma
rom the Solr - User mailing list archive at Nabble.com. -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350

Re: Special characters index

2011-03-30 Thread Markus Jelsma
; View this message in context: > http://lucene.472066.n3.nabble.com/Special-characters-index-tp2753707p2753 > 800.html Sent from the Solr - User mailing list archive at Nabble.com. -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350

Re: assit with the Clustering component in Solr/Lucene

2011-03-30 Thread Markus Jelsma
Yes, you can set engine specific parameters. Check the comments in your snippety. > Hi: > I recently included the CLustering component into Solr and updated the > requestHandler accordingly (in solrconfig.xml). Snippet of the Config for > the CLuserting: > >name="clusteringComponent" >

Re: Unexpected character '0' (code 48) in prolog;

2011-03-31 Thread Markus Jelsma
, 2011 5:21:52 AM org.apache.solr.common.SolrException log > SEVERE: Error processing "legacy" update > command:com.ctc.wstx.exc.WstxUnexpectedCharException: Unexpected character > '0' (code 48) in prolog; expected '<' at [row,col {unknown-source}]: [1,1] -- Markus Jelsma - CTO - Ope

Re: 2 index within the same Solr server ?

2011-03-31 Thread Markus Jelsma
eContainer container = super.getCoreContainer(); > >> > >> SolrCore mycore1 = container.getCore("core1"); > >> SolrCore mycore2 = container.getCore("core2"); > >> > >> So how I could index and search document within the 2 indexes? > >> > >> Thank you for your help. > >> > >> > >> 2011/3/29 Rahul Warawdekar > >> > >> > Please refer > >> > http://wiki.apache.org/solr/MultipleIndexes > >> > > >> > On 3/29/11, Amel Fraisse wrote: > >> > > Hello every body, > >> > > > >> > > Is it possible to create 2 index within the same Solr server ? > >> > > > >> > > Thank you. > >> > > > >> > > Amel. > >> > > >> > -- > >> > Thanks and Regards > >> > Rahul A. Warawdekar -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350

Re: 2 index within the same Solr server ?

2011-03-31 Thread Markus Jelsma
ment to > be indexed by both cores? > > Thank you. > Amel. > > 2011/3/31 Markus Jelsma > > > On Thursday 31 March 2011 12:58:54 Amel Fraisse wrote: > > > But for indexing and searching I've the same question: I don't know how > > > I could s

Re: how to set cookie for url requesting in stream_url

2011-03-31 Thread Markus Jelsma
clude a cookie value while requesting for the > remote_server_url. > > > Regards, > satya -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350

Re: DIH OutOfMemoryError?

2011-03-31 Thread Markus Jelsma
Try splitting the files into smaller chunks. It'll help. > Hi, > > I'm trying to index a big XML file (800Mo) using DIH, but i'm getting an > OutOfMemoryError! > > I've got 2048mo of RAM on this server, obviously it's not enough... How > much RAM is recomended for indexing big files? > > Thanks

Re: Using MLT feature

2011-04-04 Thread Markus Jelsma
afely add the doc. > > > > > > > > Is this feasible using the MLT handler? Is it a good approach? Are > > there > > > a better way to perform this comparison? > > > > > > > > Thank you for your help. > > > > > > > > Best regards, > > > > > > > > Frederico Azeiteiro -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350

Re: Using MLT feature

2011-04-04 Thread Markus Jelsma
> duplicate using a Lookup3Signature on the field defined? All duplicates > are ignored and not added to my index? > Is it so simple as that? > > Does it works even if the medianame should be an exact match (not similar > match as the headline and bodytext are)? > > Thank y

Re: Using MLT feature

2011-04-05 Thread Markus Jelsma
e frequency (1) so quant will always be 1 in this short text. That's why TextProfileSignature works less well for short texts. http://nutch.apache.org/apidocs-1.2/org/apache/nutch/crawl/TextProfileSignature.html > > Is anyone using the TextProfileSignature with success? > > Than

Re: Using MLT feature

2011-04-05 Thread Markus Jelsma
5 > > On the processor tag. > > Best regards, > Frederico > > > -Original Message- > From: Markus Jelsma [mailto:markus.jel...@openindex.io] > Sent: terça-feira, 5 de Abril de 2011 12:01 > To: solr-user@lucene.apache.org > Cc: Frederico Azeiteiro > S

Re: Indexing data with Trade Mark Symbol

2011-04-05 Thread Markus Jelsma
Any word delimiter filter will get rid of that symbol. Use a char pattern replace filter, that should work. > Use admin/analysis.jsp to see which filter is removing it. > Configure a field type appropriate to what you want to index. > > On Mon, Apr 4, 2011 at 9:55 AM, mechravi25 wrote: > > Hi,

Re: question on solr.ASCIIFoldingFilterFactory

2011-04-05 Thread Markus Jelsma
It's not the ASCII folding filter but the stemmer that's removing some trailing characters. Something you can easily spot on the analysis page. > Here is the field type definition for ‘text’ field which is what I am using > for the indexed fields. Can you guys notice any obvious filter that coul

Re: Script to remove all index.* leftovers

2011-04-05 Thread Markus Jelsma
Hi, This seems alright as it leaves the current index in place, doesn't mess with the spellchecker and leave the properties alone. But, there are two problems: 1. it doesn't take into account the commitsToKeep value set in the deletion policy, and; 2. it will remove any directory to which a cur

Re: ConcurrentLRUCache$Stats error

2011-04-05 Thread Markus Jelsma
https://issues.apache.org/jira/browse/SOLR-1797 > I'm using solr 1.4.1 and just noticed a bunch of these errors in the > solr.log file: > > SEVERE: java.util.concurrent.ExecutionException: > java.lang.NoSuchMethodError: > org.apache.solr.common.util.ConcurrentLRUCache$Stats.add(Lorg/apache/solr/c

Re: Script to remove all index.* leftovers

2011-04-06 Thread Markus Jelsma
? > > I would only run this on the slave. > > Bill > > > On Tue, Apr 5, 2011 at 2:54 PM, Markus Jelsma > > wrote: > > Hi, > > > > This seems alright as it leaves the current index in place, doesn't mess > > with the spellchecker and leave

Re: Solr: Images, Docs and Binary data

2011-04-06 Thread Markus Jelsma
> Ha, there's a binary field type?! > > I've stored binary data in an ordinary "String" field type, and it's > worked. But there were some headaches to get it to work, might have > been smoother if I had realized there was actually a binary field type. How, you can't just embed control characte

Re: Solr: Images, Docs and Binary data

2011-04-06 Thread Markus Jelsma
> Hi, your answers were really helpfull > > I was thinking in putting the base64 encoded file into a string field. But > was a little worried about solr trying to stem it or vectorize or those > stuff. String field types are not analyzed. So it doesn't brutalize your data. Better use BinaryFiel

Re: Solr: Images, Docs and Binary data

2011-04-06 Thread Markus Jelsma
> On Wed, Apr 6, 2011 at 15:31 PM, Adam Estrada > > I wanted to know how large field's size affects performance. If you use replication then it's a huge impact on performance as the data gets sent over the network. It's also a memory hog so there's less memory and more garbage collection. Ind

Re: unindexible Chars?

2011-04-06 Thread Markus Jelsma
> Once and awhile, my post.jar seems to fail on commit. Durring the commit > process, I have gotten a few errors. One is that EOF character found, and > another is that semicolon expected after &the. I also have come across a > > was expected. > > So my question is what characters do I need to st

Re: Trade Mark symbol(TM) in Index

2011-04-07 Thread Markus Jelsma
You opened the same thread this monday and got two replies. > Hi, > Has anyone indexed the data with Trade Mark symbol??...when i tried to > index, the data appears as below... I want to see the Indexed data with TM > symbol > > Indexed Data: > 79797 - Siebel Research– AI Fund, > 79797

Re: Trade Mark symbol(TM) in Index

2011-04-08 Thread Markus Jelsma
http://lucene.472066.n3.nabble.com/Indexing-data-with-Trade-Mark-Symbol- td2774421.html > Hi, > > I have to jump into this topic. > > I can not find the mentioned replies, Markus but I still noticed that > problem, too. > > What could be the cause? > > Regards, &

Re: DIH : Unexpected character '=' (code 61); expected a semi-colon after the reference for entity 'st'

2011-04-13 Thread Markus Jelsma
cordReader$Node.access$200(XPathR > ecordReader.java:202) at > org.apache.solr.handler.dataimport.XPathRecordReader.streamRecords(XPathRec > ordReader.java:184) ... 11 more > > Apr 13, 2011 4:00:25 PM org.apache.solr.update.DirectUpdateHandler2 > rollback INFO: start rollback > Apr 13, 2011 4:00:25 PM org.apache.solr.update.DirectUpdateHandler2 > rollback INFO: end_rollback -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350

<    6   7   8   9   10   11   12   13   14   15   >