Re: Whys this query not working?

2012-08-12 Thread ultranerds
Aaah thanks that makes sense. I wasn't aware of that bug. Doing the suggested
changes manually in the query works perfectly (just gotta work out how to
edit the Perl module I'm using to generate that query, so that I can use
this slightly different syntax compared to what it outputs normally)

Thanks :)

Andy



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Whys-this-query-not-working-tp4000598p4000683.html
Sent from the Solr - User mailing list archive at Nabble.com.


test

2012-08-12 Thread george123
test



--
View this message in context: 
http://lucene.472066.n3.nabble.com/test-tp4000660.html
Sent from the Solr - User mailing list archive at Nabble.com.


Re: Solritas in production

2012-08-12 Thread george123
Apologies to drag this conversation up.

I want to share my experience.

Im a code hacker, I have written about 10 lines of code in my life, but can
figure out most things and how they work, or how to get them to work.

Im looking at deploying a vertical search engine. The data is a bit messy,
but im normalising it as best I can and solr just seems to handle it so
well. Who can argue with results in miliseconds.

Now, I need to develop a front end. This is proving THE most difficult part
of my solr experience.

My options are.
1. Defailt Solritas/velocity
2. Wrap this into a drupal or similar cms with solr functionality.
3. Code this in php.

I dont have the knowledge to code in php. I cant use a cms for a variety of
reasons, most of which are related to how it is handled in the database and
the format of my data not fitting to a cms's framework, so im left to look
at point 1 again.

Im sure the knowledge in this forum is without question, but when I see
things like not production ready that doesnt mean much to me, someone
hacking out a website. Im bootstrapping so finding someone to code an option
3 for me is a bit hard, even with offshore freelance help.

If there was a set of steps to get Velocity working with my system that
would give me a solution. Alternatively, if the php (or other) code clients
coded a basic frontend template that guys like me could hack out that would
open up the solr framework to a wider range of people.

Anyway, after reading this post, and the one on this blog
http://thoughtsasaservice.wordpress.com/2012/05/10/should-you-use-solritas-on-production/
I think I am going to look down the path of this further. 

The biggest drawback to this I can see is that Velocity consumes more
memory, for me in my situation thats actually ok. Using host side tools like
apache httpd or varnish as has been suggested is something that is possible
in my situation - but a bolt on solution would be ideal. 

Hopefully my experience helps others. I wonder how many people like me
(wordpress/joomla template hackers at best) go to the apache solr project
and turn away because they cant figure out aspects, or they cant just
install a simple solution.

Dont dump velocity, it ticks so many boxes. Please add to it!



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solritas-in-production-tp3966191p4000661.html
Sent from the Solr - User mailing list archive at Nabble.com.


Re: Solritas in production

2012-08-12 Thread Erick Erickson
Just make really, really sure you don't allow queries like:
.../solr/update?stream.body=deletequery*:*/query/delete

Best
Erick

On Sun, Aug 12, 2012 at 8:42 AM, george123 daniel.tarase...@gmail.com wrote:
 Apologies to drag this conversation up.

 I want to share my experience.

 Im a code hacker, I have written about 10 lines of code in my life, but can
 figure out most things and how they work, or how to get them to work.

 Im looking at deploying a vertical search engine. The data is a bit messy,
 but im normalising it as best I can and solr just seems to handle it so
 well. Who can argue with results in miliseconds.

 Now, I need to develop a front end. This is proving THE most difficult part
 of my solr experience.

 My options are.
 1. Defailt Solritas/velocity
 2. Wrap this into a drupal or similar cms with solr functionality.
 3. Code this in php.

 I dont have the knowledge to code in php. I cant use a cms for a variety of
 reasons, most of which are related to how it is handled in the database and
 the format of my data not fitting to a cms's framework, so im left to look
 at point 1 again.

 Im sure the knowledge in this forum is without question, but when I see
 things like not production ready that doesnt mean much to me, someone
 hacking out a website. Im bootstrapping so finding someone to code an option
 3 for me is a bit hard, even with offshore freelance help.

 If there was a set of steps to get Velocity working with my system that
 would give me a solution. Alternatively, if the php (or other) code clients
 coded a basic frontend template that guys like me could hack out that would
 open up the solr framework to a wider range of people.

 Anyway, after reading this post, and the one on this blog
 http://thoughtsasaservice.wordpress.com/2012/05/10/should-you-use-solritas-on-production/
 I think I am going to look down the path of this further.

 The biggest drawback to this I can see is that Velocity consumes more
 memory, for me in my situation thats actually ok. Using host side tools like
 apache httpd or varnish as has been suggested is something that is possible
 in my situation - but a bolt on solution would be ideal.

 Hopefully my experience helps others. I wonder how many people like me
 (wordpress/joomla template hackers at best) go to the apache solr project
 and turn away because they cant figure out aspects, or they cant just
 install a simple solution.

 Dont dump velocity, it ticks so many boxes. Please add to it!



 --
 View this message in context: 
 http://lucene.472066.n3.nabble.com/Solritas-in-production-tp3966191p4000661.html
 Sent from the Solr - User mailing list archive at Nabble.com.


Re: Running out of memory

2012-08-12 Thread Alexey Serba
 It would be vastly preferable if Solr could just exit when it gets a memory
 error, because we have it running under daemontools, and that would cause
 an automatic restart.
-XX:OnOutOfMemoryError=cmd args; cmd args
Run user-defined commands when an OutOfMemoryError is first thrown.

 Does Solr require the entire index to fit in memory at all times?
No.

But it's hard to say about your particular problem without additional
information. How often do you commit? Do you use faceting? Do you sort
by Solr fields and if yes what are those fields? And you should also
check caches.


Solr becomes unresponsive while merging

2012-08-12 Thread Serdar Sahin
Hi,

We have been using Solr for last three years without any problem.

We have recently migrated our servers to a new datacenter, and our
servers have been upgraded as well. We also used this opportunity to
upgrade the software we use in our infrastructure. As a result of
this, Solr is upgraded to version 3.6.

However, we have been experiencing issues while merging. We have about
130.000 documents, and average page count is something about 80 pages.
We are running a delta indexing every hour, and about 20.000-30.000
documents get re-indexed. The reason is that we have about 80k unique
visitors, and we need to re-index deleted, added and viewed documents
(for view count). We are considering to create a new indexing format
to separate viewed documents, and run it daily to reduce number of
documents for the hourly operation. However it is not going to solve
the problem, it will just make things better. (We have workarounds
right now to prevent the issue, but no real solution)

We have no problems while indexing. However, when the merging starts,
we simply lose the connection to the server, and solr becomes
unresponsive for 7-12 minutes. Average load in the server also
increases.

So far;

- We tried changing compound file format
- We tried TieredMergePolicy, however we haven't changed
maxMergeAtOnce and segmentsPerTier. If they are going to help us, what
should be the ideal value?
- We switched back to LogByteSizeMergePolicy since it was the default
value in our old configuration but it didn't help either.

What do you suggest? How can we prevent the unresponsiveness during
merging. The reason I am saying merging is the problem is that we
experience the issue right after indexing, and it takes only 7-12
minutes. Could be something internal I do not know though. However,
the new server is quite powerful, way powerful than old server. So, it
was a bit of surprise since we haven't had any issues in the old
setup.

Thanks,