Hi All,
I'm using the build in replication with master/slave(s) Solr and the
indices are replicating just fine.
Just something troubles me:
Nothing happens in my logs/ directory ..
On the slave(s), no logs/snapshot.current file.
And on the master, nothing either appears on logs/clients/
The
Oops.
Ok my mistakes.
The logs are actually for the solr 1.3 system scripts based distribution only.
And the config files synchronize only on change ..
J.
2010/1/20 Jérôme Etévé jerome.et...@gmail.com:
Hi All,
I'm using the build in replication with master/slave(s) Solr and the
indices
Hi,
It seems this situation is caused by some No space left on device exeptions:
SEVERE: java.io.IOException: No space left on device
at java.io.RandomAccessFile.writeBytes(Native Method)
at java.io.RandomAccessFile.write(RandomAccessFile.java:466)
at
Hi,
you need to quote your phrase when you search for 'Social News':
feedClass:Social News (URI encoded of course).
otherwise your request will become (I assume you're using a standard
query parser) feedClass:Social defaultField:News . Well that's the
idea.
It should then work using the type
Hi,
How do you post your data to solr? If it's by posting XML, then it
should be properly encoded in UTF-8 (which is the XML default).
Regardless of what's in the DB (which can be a mystery with MySQL).
At query time, if the XML writer is used, then it's encoded in UTF-8.
If the json one is
/str
str name=firstLetterTitleM/str
str name=titleMix/str
/doc
On Nov 4, 2009, at 12:15 PM, Jérôme Etévé wrote:
Hi,
you need to quote your phrase when you search for 'Social News':
feedClass:Social News (URI encoded of course).
otherwise your request will become (I assume you're using
Hi,
I've got a few machines who post documents concurrently to a solr
instance. They do not issue the commit themselves, instead, I've got
autocommit set up at solr server side:
autoCommit
maxDocs5/maxDocs !-- commit at least every 5 docs --
maxTime6/maxTime !--
Hi, here's two thing that can slow down commits:
1) Autowarming the caches.
2) The Java old generation object garbage collection.
You can try:
- Turning autowarming off (set autowarmCount=0 in the caches configuration)
- If you use the sun jvm, use -XX:+UseConcMarkSweepGC to get a less
Hi All,
I'm using a multified query parser to generated weighted queries
across different fields.
For instance, perl developer gives me:
+(title:perl^10.0 keywords:perl company:perl^3.0)
+(title:developer^10.0 keywords:developer company:developer^3.0)
Either in solr 1.3 or solr 1.4 (from 12 oct
.
Jerome.
2009/10/27 Yonik Seeley yo...@lucidimagination.com:
On Tue, Oct 27, 2009 at 8:44 AM, Jérôme Etévé jerome.et...@gmail.com wrote:
I don't really get why these two tokens are subsequently put together
in a phrase query.
That's the way the Lucene query parser has always worked... phrase
0,9 10,11 12,19 20,26
10,19
Could you see a reason why title:d affaire works with textgen but
not with our type?
Thanks!
Jerome.
2009/10/27 Jérôme Etévé jerome.et...@gmail.com:
Hum,
That's probably because of our own customized types/tokenizers/filters.
I tried reindexing
Hi all,
I'm using Solr trunk from 2009-10-12 and I noticed that the QTime
result is always a multiple of roughly 50ms, regardless of the used
handler.
For instance, for the update handler, I get :
INFO: [idx1] webapp=/solr path=/update/ params={} status=0 QTime=0
INFO: [idx1] webapp=/solr
2009/10/23 Andrzej Bialecki a...@getopt.org:
Jérôme Etévé wrote:
Hi all,
I'm using Solr trunk from 2009-10-12 and I noticed that the QTime
result is always a multiple of roughly 50ms, regardless of the used
handler.
For instance, for the update handler, I get :
INFO: [idx1] webapp
Hi all,
I'm wondering where a slave pulls the files from the master on replication.
Is it directly to the index/ directory or is it somewhere else before
it's completed and gets copied to index?
Cheers!
Jerome.
--
Jerome Eteve.
http://www.eteve.net
jer...@eteve.net
Hi,
here's what you could do:
* Use multivalued fields instead of 'comma separated values', so you
won't need a separator.
* Store project identifiers in the user index.
Denormalised projects informations in a user entry will fatally need
re-indexing lot of user entries when project info
Hi All,
I'm facing a small problem with the replication handler:
After restarting my master container (tomcat),
/admin/replication/index.jsp shows me the right information,
basically the same indexversion as before the restart (no
commits/optimize have been done after restart):
Local Index
-- Forwarded message --
From: Jérôme Etévé jerome.et...@gmail.com
Date: 2009/10/16
Subject: Re: Replication filelist command failure on container restart
To: yo...@lucidimagination.com
Thanks Yonik,
It works now!
J.
2009/10/16 Yonik Seeley yo...@lucidimagination.com:
I think
Hi all,
Have you planned a release date for solr 1.4? If I understood well, it
will use lucene 2.9 release from last sept. 24th with a stable API?
Thanks.
Jerome.
--
Jerome Eteve.
http://www.eteve.net
jer...@eteve.net
Hi all,
I've got my own query parser plugin defined thanks to the queryParser tag:
queryParser name=myqueryparser class=my.package.MyQueryParserPlugin /
The QParserPlugin class has got an init method like this:
public void init(NamedList args);
Where and how do I put my args to be passed to
Hi,
- Try to let solr do the commits for you (setting up autocommit
feature). (and stop committing after inserting one document). This
should greatly improve the delays you're experiencing.
- If you do not optimize, it's normal your index size only grows.
Optimize once regularly when your load
Solr is a separate service, in the same way a RDMS is a separate service.
Whether you install it on the same machine as your webserver or not,
it's logically separated from your server.
Jerome.
2009/9/30 Claudio Martella claudio.marte...@tis.bz.it:
Kevin Miller wrote:
Does Solr have to be
Hi solr addicts,
I know there's no one size fits all set of options for the sun JVM,
but I think It'd be useful to everyone to share your tips on using the
sun JVM with solr.
For instance, I recently figured out that setting the tenured
generation garbage collection to Concurrent mark and sweep
Hi all,
I've got a solr server under significant load ( ~40/s ) and a single
process which can potentially commit as often as possible.
Typically, when it commits every 5 or 10s, my solr server slows down
quite a lot and this can lead to congestion problems on my client
side.
What would you
Hi,
You can enable/disable stemming per field type in the schema.xml, by
removing the stemming filters from the type definition.
Basically, copy your prefered type, rename it to something like
'text_nostem', remove the stemming filter from the type and use your
'text_nostem' type for your
Hi all,
I'm kind of struggling with a customized lucene.Scorer of mine, since
I use solr 1.4.
Here's the problem:
I wrote a DocSetQuery which inherit from a lucene.Query. This query
is a decorator for a lucene.Query that filters out the documents which
are not in a given set of predefined
Hi all,
I have a custom search component which uses a query I wrote.
Basically, this Query (called DocSetQuery) is a Query decorator that
skips any document which is not in a given document set. My code used
to work perfectly in solr 1.3 but in solr 1.4, it seems that my
DocSetQuery has lost all
.
Shortly (when Solr trunk updates Lucene), Solr will go back to just
createWeight and weight.
The main change that will be left is that Weight will be an abstract class
rather than an interface.
--
- Mark
http://www.lucidimagination.com
Jérôme Etévé wrote:
Hi all,
I have a custom search
, or for the primitive Query
it rewrites itself too).
--
- Mark
http://www.lucidimagination.com
Jérôme Etévé wrote:
Hi Mark,
Thanks for clarifying this. So should I keep both sets of method
implemented? I guess it won't hurt when solr trunk will use the
updated version of lucene without those
performance.
Do note that trunk (soon Solr 1.4) has dramatically improved faceting
performance.
Erik
On Aug 12, 2009, at 1:30 PM, Jérôme Etévé wrote:
Hi everyone,
I'm using some faceting on a solr index containing ~ 160K documents.
I perform facets on multivalued string fields
Hi everyone,
I'm using some faceting on a solr index containing ~ 160K documents.
I perform facets on multivalued string fields. The number of possible
different values is quite large.
Enabling facets degrades the performance by a factor 3.
Because I'm using solr 1.3, I guess the facetting
Hi,
I'm running solr 1.3 with java -version java version 1.6... .
No problem to report.
Cheers.
J
2009/8/12 vaibhav joshi callvaib...@hotmail.com:
Hi
I am using Solr 1.3 ( official released version) and JDk1.5. My company is
moving towards upgrading all systems to JDK1.6. is it safe to
Hi all,
I'd like to have a string type which is synonym aware at query time.
Is it ok to have something like that:
fieldType name=sastring class=solr.StrField
analyzer type=query
tokenizer class=solr.KeywordTokenizerFactory/
filter class=solr.SynonymFilterFactory
that, it will be used both at index and
query time.
Otis
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
- Original Message
From: Jérôme Etévé jerome.et...@gmail.com
To: solr-user@lucene.apache.org
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
- Original Message
From: Jérôme Etévé jerome.et...@gmail.com
To: Otis Gospodnetic otis_gospodne...@yahoo.com
Cc: solr-user@lucene.apache.org
Sent: Tuesday, August 4, 2009 12:39:33 PM
Subject: Re: Synonym aware string field
Hi all,
Is there a way to enable faceting when using a more like this handler?
I'd like to have facets from my similar documents.
Cheers !
J.
--
Jerome Eteve.
Chat with me live at http://www.eteve.net
jer...@eteve.net
Hi All,
I'm wondering if it's possible to post documents to solr in JSON format.
JSON is much faster than XML to get the queries results, so I think
it'd be great to be able to post data in JSON to speed up the indexing
and lower the network load.
All the best !
Jerome Eteve.
--
Jerome
Hi,
Nope, I'm not using solrj (my client code is in Perl), and I'm with solr 1.3.
J.
2009/7/30 Shalin Shekhar Mangar shalinman...@gmail.com:
On Thu, Jul 30, 2009 at 8:31 PM, Jérôme Etévé jerome.et...@gmail.com
wrote:
Hi All,
I'm wondering if it's possible to post documents to solr
Hi all,
I don't know if it does the same from everyone, but when I use the
reply function of my mail agent, it sets the recipient to the user who
sent the message, and not the mailing list.
So it's quite annoying cause I have to change the recipient each time
I reply to someone on the list.
2009/7/30 Erik Hatcher e...@ehatchersolutions.com:
On Jul 30, 2009, at 1:44 PM, Jérôme Etévé wrote:
Hi all,
I don't know if it does the same from everyone, but when I use the
reply function of my mail agent, it sets the recipient to the user who
sent the message, and not the mailing list
Hi All,
I'm planning to have a certain number of processes posting
independently in a solr instance.
This instance will solely act as a master instance. No clients queries on it.
Is there a problem if i set maxWarmingSearchers to something like 30 or 40?
Also, how do I disable the cache
Hi All,
I've got here a small problem about replication.
Let's say I post a document on the master server, and the slaves do
a snappuller/installer via crontab every 1 minutes.
Then between in average 30 seconds, all my search servers are not
synchronized.
Is there a way to improve
Hi Everyone,
I'm running solr 1.3 and I was wondering if there's a problem with
running the snapshot script concurrently .
For instance, I have a cron job which performs a
snappuller/snapinstaller every minute on my slave servers. Sometime
(for instance after an optimize), the snappuller can
Hi !
Is there any primary table in your view with a unique single key
you could use ?
J.
2009/5/11 jcott28 jcot...@yahoo.com:
I have a case where I would like a solr index created which disables the
unique-key option.
I've tried commenting out the uniqueKey option and that just spits
Seeley yo...@lucidimagination.com wrote:
Looks like cache autowarming.
If you have statically defined warming queries in solrconfig.xml, you
could try setting autowarmCount=0 for all the caches.
-Yonik
http://www.lucidimagination.com
On Tue, Mar 3, 2009 at 2:37 PM, Jérôme Etévé jerome.et
On Wed, Mar 4, 2009 at 1:21 PM, Yonik Seeley yo...@lucidimagination.com wrote:
On Wed, Mar 4, 2009 at 5:25 AM, Jérôme Etévé jerome.et...@gmail.com wrote:
Great,
It went down to less than 10 secs now :)
What I don't really understand is that my autowarmCount were pretty
low ( like 128
Dear solr fans,
I have a solr index of roughly 8M docs and I have here a little
problem when I commit some insertion into it.
The insert itself is very fast, but my commit takes 163 seconds.
Here's the solr trace the commit leaves:
INFO: start
Hi fellow Solr fans,
I'm setting up some collection distribution along with multicore
solr . I'm using version 1.3
I have no problem with the snapshooter, since this can be set within
each core in solrconfig.xml.
My question is more about the rsyncd .
The rsyncd-start creates a
Hi all,
In solr.xml ( /lucene/solr/trunk/src/webapp/web/WEB-INF/web.xml
),it's written that
It is unnecessary, and potentially problematic, to have the SolrDispatchFilter
configured to also filter on forwards. Do not configure
this dispatcher as dispatcherFORWARD/dispatcher.
The
at 6:24 PM, Jérôme Etévé jerome.et...@gmail.com wrote:
Maybe there's an 'internal query' concept in j2ee that could be a workaround ?
I'm not really a j2ee expert ..
Jerome.
On Wed, Dec 17, 2008 at 5:09 PM, Smiley, David W. dsmi...@mitre.org wrote:
This bothers me too. I find it really
Hi Lance,
Can you tell us what's this parameter and how to set it ?
I'm also stucked with the same problem :(
Thanks !!
Jerome
On Mon, Sep 8, 2008 at 6:02 PM, Lance Norskog goks...@gmail.com wrote:
You can give a default core set by adding a default parameter to the query
in
Hi everyone,
I'm wondering if the MoreLikeThis handler takes the boost function
parameter into account for the scoring (hence the sorting I guess) of
the similar documents it finds.
Thanks for your help !
Jerome.
--
Jerome Eteve.
Chat with me live at http://www.eteve.net
[EMAIL PROTECTED]
Hi there,
I developed a personalized SearchComponent in which I'm building a
docset from a personalized Query, and a personalized Priority Queue.
To be short, I'm doing that (in the process method) :
HitCollector hitCol = new HitCollector() {
@Override
public void
next(Token result) there is no information about
the location inside the stream.
I can read characters from the input Reader, but couldn't find a way to know
if it's the beginning of the input or not.
-J
On Fri, Nov 7, 2008 at 6:13 AM, Jérôme Etévé [EMAIL PROTECTED] wrote:
Hi,
I think you
Hi,
For batch indexing, what you could do is to use two core. One in
production and one used for your update.
Once your update core is build (delete *:* plus batch insert) , you
can swap the cores to put it in production:
Hi,
I think you could implement your personalized tokenizer in a way it
changes its behaviour after it has delivered X tokens.
This implies a new tokenizer instance is build from the factory for
every string analyzed, which I believe is true.
Can this be confirmed ?
Cheers !
Jerome.
On
Hi all,
In my code, I'd like to keep a subset of my 14M docs which is around
100k large.
What is according to you the best option in terms of speed and memory usage ?
Some basic thoughts tells me the BitDocSet should be the fastest for
lookup, but takes ~ 14M * sizeof(int) in memory,
Hi everyone,
I'm implementing a search component inherited from SearchComponent .
This component has to build a data structure from the index. Like in
the SpellChecker, I trigger this building by giving a special argument
at query time (from the process method) and I'm using the searcher I
Hi there,
Are you sure you did a commit after your insertion ?
On Fri, Oct 24, 2008 at 8:11 AM, sunnyfr [EMAIL PROTECTED] wrote:
Even that doesn't work,
How can I check properly, I did insert one document but I can't get it back
???
Feak, Todd wrote:
Unless q=ALL is a special query I
Great, it works now.
Thanks !
J
On Fri, Oct 24, 2008 at 4:45 PM, Yonik Seeley [EMAIL PROTECTED] wrote:
On Fri, Oct 24, 2008 at 8:21 AM, Jérôme Etévé [EMAIL PROTECTED] wrote:
I though it'd be ok to trigger this the very first time the process
method is called by doing something like
Hi,
See
http://java.sun.com/j2se/1.3/docs/guide/intl/encoding.doc.html
and
http://java.sun.com/j2se/1.4.2/docs/api/java/lang/String.html#getBytes(java.lang.String)
Also note that you cannot transform a latin1 string in a utf-8
string. What you can do
is to decode a latin1 octet
On Tue, Oct 7, 2008 at 12:56 AM, Chris Hostetter
[EMAIL PROTECTED] wrote:
: req.getSchema().getQueryAnalyzer();
:
: I think it's in this analyzer that the undefined field error happens
: (because for instance the field 'foo' doesn't exists in the schema,
: and so it's impossible to find a
[EMAIL PROTECTED] wrote:
On Tue, Sep 30, 2008 at 2:42 PM, Jérôme Etévé [EMAIL PROTECTED] wrote:
But still I have an error from the webapp when I try to query my
schema with non existing fields in my query ( like foo:bar ).
I'm wondering if the query q is parsed in a very simple way somewhere
Hi All,
I wrote a customized query parser which discards non-schema fields
from the query (I'm using the schema field names from
req.getSchema().getFields().keySet() ) .
This parser works fine in unit tests.
But still I have an error from the webapp when I try to query my
schema with non
Hello all.
I'm using a multicore installation and I've got a small issue with
the loading of our customized jars.
Let's say I've got a class my.company.MyAnalyzer which is distributed
in a jar called company-solr.jar
If I put this jar in the lib directory, at the solr home like this:
My mistake,
Using the sharedLib=lib/ attribute in the solr tag of solr.xml
solved the problem.
J.
On Mon, Sep 29, 2008 at 2:43 PM, Jérôme Etévé [EMAIL PROTECTED] wrote:
Hello all.
I'm using a multicore installation and I've got a small issue with
the loading of our customized jars
Hi everyone,
I'm planning to use the multicore cause it seems more convenient
than having multiple instances of solr in the same container.
I'm wondering if it's possible to query different cores ( hence
different schemas / searchers ... indices !) from a customized
SolrRequestHandler to
Hi,
Here's what I've got (multiplesolr instance within the same tomcat server)
In
/var/tomcat/conf/Catalina/localhost/
For an instance 'foo' :
foo.xml :
Context path=foo docBase=/var/tomcat/solrapp/solr.war debug=0
crossContext=true
Environment name=solr/home type=java.lang.String
On 10/9/07, Chris Laux [EMAIL PROTECTED] wrote:
Jérôme Etévé wrote:
[...]
/var/solr/foo/ is the solr home for this instance (where you'll put
your schema.xml , solrconfig.xml etc.. ) .
Thanks for the input Jérôme, I gave it another try and discovered that
what I was doing wrong
If I understand, you want to keep the raw html code in solr like that
(in your posting xml file):
field name=storyFullText
html/html
/field
I think you should encode your content to protect these xml entities:
- lt;
- gt;
- quot;
- amp;
If you use perl, have a look at HTML::Entities.
By design, it's not very efficient to ask for a large number of
results with solr/lucene. I think you will face performance and memory
problems if you do that.
On 9/24/07, Thorsten Scherler [EMAIL PROTECTED] wrote:
On Mon, 2007-09-24 at 16:29 +0530, Roopesh P Raj wrote:
Hi Roopesh,
I am
Hi all
I've got a problem here with the '-field:[* TO *]' syntax. It doesn't
seem to work as expected (see
http://wiki.apache.org/solr/SolrQuerySyntax ).
My request is 'word -fieldD:[* TO *]' and the debugQuery=1 solr option
shows that it's properly transformed as :
+(fieldA:chef^10.0
On 8/21/07, Chris Hostetter [EMAIL PROTECTED] wrote:
: I'm wondering what's the best way to completely change a big index
: without loosing any requests.
use the snapinstaller script -- or adopt the same atomic copying approach
it uses.
I'm having a look :)
: - Between the two mv's,
You need to encode your html content so it can be include as a normal
'string' value in your xml element.
As far as remember, the only unsafe characters you have to encode as
entities are:
- lt;
- gt;
- quote;
- amp;
(google xml entities to be sure).
I dont know what language you use , but
Hi !
Is there a way to use a MMapDirectory instead of FSDirectory within Solr ?
Our index is quite big and It takes a long time to go up in the OS
cached memory. I'm wondering if an MMapDirectory could help to have
our data in memory quicker (our index on disk is bigger than our
memory
Hi all,
I'm wondering what's the best way to completely change a big index
without loosing any requests.
That's how I do at the moment:
solr index is a soft link to a directory dir.
When I want to install a new index (in dir.new), I do a
mv dir dir.old ; mv dir.new dir
Then I ask for a
Hi all !
I need a new feature in solr : to allow the configuration of the
IndexSearcher class in the schema configuration to override the lucene
IndexSearcher .
I noticed that there's only one point in the code where the searcher is built:
code
in org/apache/solr/search/SolrIndexSearcher.java:
in the solr jira:
https://issues.apache.org/jira/browse/SOLR-288
but I didn t find the way to upload the patch.
Thanks for your comments.
Jerome.
On 7/5/07, Jérôme Etévé [EMAIL PROTECTED] wrote:
Hi all !
I need a new feature in solr : to allow the configuration of the
IndexSearcher class in the schema
Hi ,
when we use DisMaxQueryHandler, queries that includes specific
fields which are not part of the boost string doesn't seem to work.
For instance, If the boost string ( qf ) is 'a^3 b^4' and
my query is 'term +c:term2' , it doesnt produce any result.
Am I using this QueryHandler the bad way
Hi,
is there a way to avoid going to the web interface to set up the solr
log level ?
I'm also a bit confused about the INFO log level. Actually it's very
nice to see some startup info about the schema , solr home setting,
customize modules loaded .. But also this INFO log levels gives two
On 6/29/07, Chris Hostetter [EMAIL PROTECTED] wrote:
: Hi,
: is there a way to avoid going to the web interface to set up the solr
: log level ?
he web intrface for tweaking the log level is actually a miss-feature in
my opinion ... it's a handy way to quickly crank the logging level up if
Hi,
Solr uses a default query parser which is a SolrQueryParser based on
a org.apache.lucene.queryParser.QueryParser;
I wonder which is the best way to make the IndexSchema use some kind
of MultifieldSolrQueryParser which could be based on a
org.apache.lucene.queryParser.MultiFieldQueryParser
On 6/28/07, Yonik Seeley [EMAIL PROTECTED] wrote:
On 6/28/07, Daniel Alheiros [EMAIL PROTECTED] wrote:
I'm in trouble now about how to issue queries against Solr using in my q
parameter content in Russian (it applies to Chinese and Arabic as well).
The problem is I can't send any Russian
2007/6/27, Yonik Seeley [EMAIL PROTECTED]:
It would be helpful if you could try out the patch at
https://issues.apache.org/jira/browse/SOLR-276
-Yonik
I just tryed it out and it works. json output is now as fast as xml !
Well done :) thank you !
J.
--
Jerome Eteve.
[EMAIL PROTECTED]
Hi all.
I'm currently running some tests with solr on a small index and I
noticed a big difference on the response time of queries depending on
the use of XML or json as a response format.
In average, my test queries (including http connections open and close
) takes 6 ms to perform when I ask
On 6/26/07, Yonik Seeley [EMAIL PROTECTED] wrote:
On 6/26/07, Jérôme Etévé [EMAIL PROTECTED] wrote:
I'm currently running some tests with solr on a small index and I
noticed a big difference on the response time of queries depending on
the use of XML or json as a response format
85 matches
Mail list logo