Re: Is this too much time for full Data Import?

Hello, Does your indexer utilize CPU/IO? - check it by iostat/vmstat. If it doesn't, take several thread dumps by jvisualvm sampler or jstack, try to understand what blocks your threads from progress. It might happen you need to speedup your SQL data consumption, to do this, you can enable

Re: Recovery problem in solrcloud

Aug 06, 2012 10:05:55 AM org.apache.solr.common.SolrException log SEVERE: null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space at org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:456) at

Re: Does Solr support 'Value Search'?

Hello, Have you checked http://lucidworks.lucidimagination.com/display/lweug/Wildcard+Queries ? On Wed, Aug 8, 2012 at 12:56 AM, Bing Hua bh...@cornell.edu wrote: Hi folks, Just wondering if there is a query handler that simply takes a query string and search on all/part of fields for field

Re: Connect to SOLR over socket file

On 07.08.2012 21:43, Jason Axelson wrote: Hi, Is it possible to connect to SOLR over a socket file as is possible with mysql? I've looked around and I get the feeling that I may be mi-understanding part of SOLR's architecture. Any pointers are welcome. Thanks, Jason Hi Jason, not that I

Designing an index with multiple entity types, sharing field names across entity-types.

My question stems from a vague memory of reading somewhere that Solr's search performance depends on how the total number of 'terms' there are in all in a field that is searched upon. I'm setting up an index core for some autocomplete boxes on my site. There is a search box for each facet group

Re: Designing an index with multiple entity types, sharing field names across entity-types.

To clarify a wee bit more. I'm wondering the performance impact on single-entity queries if I use common field names. eg. 'name' field for all entity types. 'Author' 'Book' together make up for 200,000+ 'name' values. Will this affect anything if I search over 'Category'? Will using

Paoding analyzer with solr for chinese

Hi All, As said in this blog sitehttp://java.dzone.com/articles/indexing-chinese-solr that paoding analyzer is much better for chinese text, I was trying to implement it to get accurate results for chinese text. I followed the instruction specified in the below sites

Re: Recovery problem in solrcloud

Stack trace looks normal - it's just a multi-term query instantiating a bitset. The memory is being taken up somewhere else. How many documents are in your index? Can you get a heap dump or use some other memory profiler to see what's taking up the space? if I stop query more then ten minutes,

Problems with EDisMax field aliases for multiple fields on Solr 3.6.1

Hey, I'm trying to use field aliases that reference multiple fields on Solr 3.6.1 (1362471) as stated in the EDisMax documentation (http://wiki.apache.org/solr/ExtendedDisMax#Field_aliasing_.2BAC8_renaming). If I use an alias for a single field, everything is fine, but once I use an alias

numFound changes on changing start and rows

Hi, We are using Solr3.6 and 2 shards, we are noticing that when we fire a query with start as 0 and rows X the total numFound and the total numFound changes when we fire the same exact query with start as y and rows X. For example. First time query=abcstart=0rows=4000 numFound-

Re: numFound changes on changing start and rows

We've noticed some pretty non-deterministic behavior with sharded setups as well. One thing we've noticed is that a query server can hang on to the set of document ids that correspond to a given query even if caching is off, which results in some weird behavior, such as a query like:

Re: numFound changes on changing start and rows

Sorry, in my time range example, I forgot to mention that you can repeatedly execute the 8 hour query and receive no results, even after the 7 hour query retrieves them. Kind of an important detail to not forget. :) Michael Della Bitta Appinions

Re: Is this too much time for full Data Import?

Pranav, If possible, you may wish to consider moving a job this large outside of DataImportHandler to a standalone program, as the SQL processing is somewhat limited by the N+1 subselects problem. Michael Della Bitta Appinions | 18 East 41st St.,

Re: Problems with EDisMax field aliases for multiple fields on Solr 3.6.1

Hi, It's hard to see what's going on without knowing more about your schema and documents. Also, it would be more helpful if you could paste the parsedquery part of the DebugQuery, where you actually see how the query was interpreted. Your query syntax looks correct, and I just verified that

Re: Problems with EDisMax field aliases for multiple fields on Solr 3.6.1

Jan, I did notice that you used a space rather than a comma in the alias field list. The wiki does indicate comma (which is what Nils used), but... who knows. I haven't checked the code yet. -- Jack Krupansky -Original Message- From: Jan Høydahl Sent: Wednesday, August 08, 2012 9:03

Re: Is this too much time for full Data Import?

9m*15 - that's a lot of queries (400 QPS). I would try reduce the number of queries: 1. Rewrite your main (root) query to select all possible data * use SQL joins instead of DIH nested entities * select data from 1-N related tables (tags, authors, etc) in the main query using GROUP_CONCAT

getting empty result but numFound 0

Hello List, i'm evaluate solr 4 / solrCloud and got the following problem. I've indexed ~1.5M Documents but the docs section in the response is always empty. The response for the *:* query looks like this: { responseHeader:{ status:500, QTime:12, params:{ fl:title,img,

RE: getting empty result but numFound 0

The status is 500, check your logs for some errors. -Original message- From:Rafael Pappert r...@fwpsystems.com Sent: Wed 08-Aug-2012 15:49 To: solr-user@lucene.apache.org Subject: getting empty result but numFound gt; 0 Hello List, i'm evaluate solr 4 / solrCloud and got the

Re: Problems with EDisMax field aliases for multiple fields on Solr 3.6.1

Thanks for the quick replies. Jack was right, I switched to space as separator and it works. 2) f.name.qf=last_name_text,first_name_text - returns 0 results, debug shows str name=f.name.qflast_name_text,first_name_text/str +DisjunctionMaxQuery((last_name_text,first_name_text:maier)) 8)

Setting df (default field) from solrj?

I see in Solr 4 the defaultSearchField tag in schema.xml has been deprecated. I was looking in the Solrj API and I don't see a method for setting the default field on a SolrQuery object. This is basically what the code looks like now (stripped down): solrQuery = SolrQuery.new()

Re: Does Solr support 'Value Search'?

Thanks for the response but wait... Is it related to my question searching for field values? I was not asking how to use wildcards though. -- View this message in context: http://lucene.472066.n3.nabble.com/Does-Solr-support-Value-Search-tp3999654p3999817.html Sent from the Solr - User

Re: Setting df (default field) from solrj?

You should simply set the default value for the df request parameter in your Solr request handlers in solrconfig.xml. It is set to text out of the box, but you can set it to your desired field. If you still want to set/override df from SolrJ anyway, use the SolrQuery.setParam method:

Re: Problems with EDisMax field aliases for multiple fields on Solr 3.6.1

There is an open Solr issue to allow commas in lists everywhere, but even in 4.0 space is still the delimiter for field name boost lists (qf and f.field-name.qf). I'll update the wiki. -- Jack Krupansky -Original Message- From: Nils Kaiser Sent: Wednesday, August 08, 2012 9:56 AM

HTTP Basic Authentication with HttpSolrServer

I have protected my solr server with basic authentication. Now I want to connect to it using SOLRJ. CommonsHttpSolrServer is now deprecated, so I try to use HttpSolrServer, but I fail to send credentials. If I put them to the url, I get 401 (http://user:passw...@example.com/solr). I tried this:

Re: HTTP Basic Authentication with HttpSolrServer

Villam, this is a question for httpclient, I think you want to enable preemptive authentication so as to avoid the need to repeat the query after the unauthorized response is sent. http://hc.apache.org/httpclient-3.x/authentication.html#Preemptive_Authentication paul Le 8 août 2012

Solr makes long requests about once a minute

I'm having a problem with Solr under Tomcat unexpectedly taking a long time to respond to queries. As part of some stress testing, I wrote a bot that just does random word searches on my Solr install, and my responses typically come back in 10-50 ms. The queries are just 1-3 random words from

Re: getting empty result but numFound 0

status:500 means there was probably an exception on the server. Check the Solr log file for details. -- Jack Krupansky -Original Message- From: Rafael Pappert Sent: Wednesday, August 08, 2012 9:20 AM To: solr-user@lucene.apache.org Subject: getting empty result but numFound 0 Hello

Re: numFound changes on changing start and rows

: We are using Solr3.6 and 2 shards, we are noticing that when we fire a query : with start as 0 and rows X the total numFound and the total numFound changes : when we fire the same exact query with start as y and rows X. The only situation where i've ever heard of this happening is when

Re: HTTP Basic Authentication with HttpSolrServer [solved]

You're partly right. The solution in the link was for CommonsHttpSolrServer, it does work for HttpSolrServer, but the principle is the same. Actually, I found solution for the new HttpClient here:

Re: numFound changes on changing start and rows

Our documents are keyed with UUIDs, and we shard chronologically. The write events are issued as part of a SQS queue that only allows one reader to see the message. I think it's pretty unlikely that we have more than one document with the same uniquekey. I can actually prove this if it will help

Re: Solr makes long requests about once a minute

Hi, Andy, What version of Solr are you running and what Directory implementation are you using? How much RAM does your system have, and how much is available for use by Solr? Michael Della Bitta Appinions | 18 East 41st St., Suite 1806 | New

RE: numFound changes on changing start and rows

I can cross check our shards once again, but I am sure this is not the case. Regards, Rohit Mobile: +91-9901768202 -Original Message- From: Chris Hostetter [mailto:hossman_luc...@fucit.org] Sent: 08 August 2012 21:04 To: solr-user@lucene.apache.org Subject: Re: numFound changes on

Re: Solr makes long requests about once a minute

On Aug 8, 2012, at 10:53 AM, Michael Della Bitta wrote: What version of Solr are you running and what Directory implementation are you using? How much RAM does your system have, and how much is available for use by Solr? Solr 3.6.0 I don't know what directory implementation means. Are you

Re: Solr makes long requests about once a minute

StandardDirectoryFactory gets us partway there, but that's actually a class that chooses an appropriate implementation at runtime based on the parameters of the system it's being run on. If you go to the status page off of the admin page and do a find on readerDir, I'm guessing you'll see

Re: Solr makes long requests about once a minute

Check the Solr log file and see if something is happening at those slow queries. Maybe an auto-commit? -- Jack Krupansky -Original Message- From: Andy Lester Sent: Wednesday, August 08, 2012 11:30 AM To: solr-user@lucene.apache.org Subject: Solr makes long requests about once a

Re: Recovery problem in solrcloud

There are 400 million documents in a shard, a document is less then 1 kb. the data file _**.fdt is 149g. Does the recovering need large memory in downloading or after downloaded? I find some log before OOM as below: Aug 06, 2012 9:43:04 AM org.apache.solr.core.SolrCore execute INFO: [blog]

search on default field returns less documents

Hi All we have two fields: field name=doc type=textgen indexed=true stored=true required=false / field name=text type=textgen indexed=true stored=false multiValued=true / 'text' is our default field: defaultSearchFieldtext/defaultSearchField we

max connections in CloudSolrServer

I see that in other constructors you can specify an HttpClient to be used, but I don't see this same option for the CloudSolrServer. Is there a way to say the maximum number of connections that should be used for CloudSolrServer? What is the current number that is supported?

/solr/admin/stats.jsp null pointer exception

New install of Solr 3.6.1, getting a Null Pointer Exception when trying to access admin/stats.jsp: record date2012-08-08T17:55:09/date millis138509624/millis sequence694/sequence loggerorg.apache.solr.servlet.SolrDispatchFilter/logger levelSEVERE/level

Re: Does Solr support 'Value Search'?

Ok. It seems to me you can configure http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.WordDelimiterFilterFactoryfor index-time to produce admin term from all your docs above, after that you'll be able to match by simple term query. Is it what are you looking for? On Wed, Aug 8,

Re: No search result is returned using Solr 4

Hi Engy, Can you able to get the result for q=*.* ? And if so can you check where query is parsing properly by add a param debug=true . Syed Abdul kather send from Samsung S3 On Aug 8, 2012 4:30 PM, engy.morsy [via Lucene] ml-node+s472066n3999741...@n3.nabble.com wrote: Hi, I downloaded

Re: Setting df (default field) from solrj?

Perfect. Thanks! Jack Krupansky-2 wrote You should simply set the default value for the df request parameter in your Solr request handlers in solrconfig.xml. It is set to text out of the box, but you can set it to your desired field. If you still want to set/override df from SolrJ

Re: Does Solr support 'Value Search'?

Not quite understand but I'd explain the problem I had. The response would contain only fields and a list of field values that match the query. Essentially it's querying for field values rather than documents. The underlying use case would be, when typing in a quick search box, the drill down menu

Re: search on default field returns less documents

Default search field handling changed in Solr 3.6. Which release of Solr are you using? In Solr 3.6, the df request parameter in your query request handler overrides the deprecated defaultSearchField. The out of the box default for df is text, which should match your schema, but... better to

Re: Syntax for parameter substitution in function queries?

Thanks very much; that does indeed work as I'd hoped/expected. On 7 August 2012 17:12, Yonik Seeley yo...@lucidimagination.com wrote: On Tue, Aug 7, 2012 at 3:01 PM, Timothy Hill timothy.d.h...@gmail.com wrote: Hello, all ... According to

Re: Configuration for distributed search

: This command to each shard returns one document from each shard. : curl 'http://localhost:8983/solr/select?debugQuery=trueindent=trueq=conway : curl 'http://localhost:7574/solr/select?debugQuery=trueindent=trueq=conway : : This distributed search command returns 0 documents: What do those two

Re: /solr/admin/stats.jsp null pointer exception

: New install of Solr 3.6.1, getting a Null Pointer Exception when trying to : access admin/stats.jsp: : org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228) : at : org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) : Caused by:

Re: Multiple Embedded Servers Pointing to single solrhome/index

No, you can only have one program controlling an index. This will not work! You should use a primary/failover technique where one program does all of the indexing, and then another program is the fallback for the first indexer. On Tue, Aug 7, 2012 at 7:31 AM, Bing Hua bh...@cornell.edu wrote:

Limit on SOLR Cores

Hi Guys, I've come across a use case where I've to keep separate indexes for multiple tanents. Data directory of each tenant should be different but SOLR server instance has same schema and configuration for all the tenants. Tenants in our case can be added dynamically. I know that I can handle

Re: max connections in CloudSolrServer

On Wed, Aug 8, 2012 at 1:55 PM, Jamie Johnson jej2...@gmail.com wrote: I see that in other constructors you can specify an HttpClient to be used, but I don't see this same option for the CloudSolrServer. You can pass a LBHttpSolrServer, which you can init with an HttpClient. Or you can use

Using SolrCloud with non string type id field?

Just curious if anyone wants to come forward as someone using SolrCloud with a non string based unique field? That is the default, so if you did not change it, you are using the string type. We are considering a change to how we handle hashing that would be back compat for the string type, but

Re: error message in solr logs

: Lately we are noticing below exception in our solr logs. This happens : sometimes once or twice a day on a few cores. the error you are seing here is a really low level HTTP communications error, below hte level of solr... : Caused by: java.io.IOException: Invalid chunk header : at :

Re: Paoding analyzer with solr for chinese

Hi All, Any reply on this? On Wed, Aug 8, 2012 at 3:23 PM, Rajani Maski rajinima...@gmail.com wrote: Hi All, As said in this blog sitehttp://java.dzone.com/articles/indexing-chinese-solr that paoding analyzer is much better for chinese text, I was trying to implement it to get

Re: question(s) re lucene spatial toolkit aka LSP aka spatial4j

Hi! Sorry for the belated response; my google alerts didn't kick in for some weird reason until you posted to the lucene dev list. solr-user wrote hopefully someone is using the lucene spatial toolkit aka LSP aka spatial4j, and can answer this question we are using this spatial tool for