Re: 400 Error adding field 'tags'='[a,b,c]'
left. the $20 under the tile with the hand prints. thx
On Tuesday, March 13, 2012, jlark alpti...@hotmail.com wrote:
Interestingly I'm getting this on other fields now.
I have the fieldfield name=name type=text_general indexed=true
stored=true /
which is copied to text copyField source=name dest=text/
and my text field is simplyfield name=text type=text_general
indexed=true stored=true /
I'm feedin my test document
{url : TestDoc2, title : another test,
ptag:[a,b],name:foo
bar},
and when I try to feed I get.
HTTP request sent, awaiting response... 400 ERROR: [doc=TestDoc2] Error
adding field 'name'='foo bar'
If I remove the field from the document though it works fine.
I'm wondering if there is a set of reserved names that I'm using at this
point.
Jus twhish there was a way to get more helpfull error messages.
Thanks for the help.
Alp
--
View this message in context:
http://lucene.472066.n3.nabble.com/400-Error-adding-field-tags-a-b-c-tp3823853p3824126.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Speeding up indexing
Many thanks for the response. Here is the revised questions: For example if I have N processes that are producing documents to index: 1. Should I have them simultaneously submit documents to Solr (will this improve the indexing throughput)? 2. Is there anything I can do Solr configuration wise that will allow me to speed up indexing 3. Is there an architecture where I can have two (or more) solr server do indexing in parallel Thanks. On Mon, Feb 27, 2012 at 1:46 PM, Erik Hatcher erik.hatc...@gmail.comwrote: Yes, absolutely. Parallelizing indexing can make a huge difference. How you do so will depend on your indexing environment. Most crudely, running multiple indexing scripts on different subsets of data up to the the limitations of your operating system and hardware is how many do it. SolrJ has some multithreaded facility, as does DataImportHandler. Distributing the indexing to multiple machines, but pointing all to the same Solr server, is effectively the same as multi-threading it push documents into Solr from wherever as fast as it can handle it. This is definitely how many do this. Erik On Feb 27, 2012, at 13:24 , Memory Makers wrote: Hi, Is there a way to speed up indexing by increasing the number of threads doing the indexing or perhaps by distributing indexing on multiple machines? Thanks.
Re: Speeding up indexing
A quick add on to this -- we have over 30 million documents. I take it that we should be looking @ Distributed Solr? as in http://www.lucidimagination.com/content/scaling-lucene-and-solr#d0e344 Thanks. On Mon, Feb 27, 2012 at 2:33 PM, Memory Makers memmakers...@gmail.comwrote: Many thanks for the response. Here is the revised questions: For example if I have N processes that are producing documents to index: 1. Should I have them simultaneously submit documents to Solr (will this improve the indexing throughput)? 2. Is there anything I can do Solr configuration wise that will allow me to speed up indexing 3. Is there an architecture where I can have two (or more) solr server do indexing in parallel Thanks. On Mon, Feb 27, 2012 at 1:46 PM, Erik Hatcher erik.hatc...@gmail.comwrote: Yes, absolutely. Parallelizing indexing can make a huge difference. How you do so will depend on your indexing environment. Most crudely, running multiple indexing scripts on different subsets of data up to the the limitations of your operating system and hardware is how many do it. SolrJ has some multithreaded facility, as does DataImportHandler. Distributing the indexing to multiple machines, but pointing all to the same Solr server, is effectively the same as multi-threading it push documents into Solr from wherever as fast as it can handle it. This is definitely how many do this. Erik On Feb 27, 2012, at 13:24 , Memory Makers wrote: Hi, Is there a way to speed up indexing by increasing the number of threads doing the indexing or perhaps by distributing indexing on multiple machines? Thanks.
performance - dynamic fields versus static fields
Hi, Is there a handy resource on the: a. performance of: dynamic fields versus static fields b. other pros-cons? Thanks.
Re: Is SQL Like operator feature available in Apache Solr query
Eric, NGrams could you elaborate on that ? -- haven't seen that before. Thanks. On Tue, Nov 1, 2011 at 11:06 AM, Erick Erickson erickerick...@gmail.comwrote: NGrams are often used in Solr for this case, but they will also add to your index size. It might be worthwhile to look closely at your user requirements before going ahead and supporting this functionality Best Erick 2011/11/1 François Schiettecatte fschietteca...@gmail.com: Kuli Good point about just tokenizing the fields :) I ran a couple of tests to double-check my understanding and you can have a wildcard operator at either or both ends of a term. Adding ReversedWildcardFilterFactory to your field analyzer will make leading wildcard searches a lot faster of course but at the expense of index size. Cheers François On Nov 1, 2011, at 9:07 AM, Michael Kuhlmann wrote: Hi, this is not exactly true. In Solr, you can't have the wildcard operator on both sides of the operator. However, you can tokenize your fields and simply query for Solr. This is what's Solr made for. :) -Kuli Am 01.11.2011 13:24, schrieb François Schiettecatte: Arshad Actually it is available, you need to use the ReversedWildcardFilterFactory which I am sure you can Google for. Solr and SQL address different problem sets with some overlaps but there are significant differences between the two technologies. Actually '%Solr%' is a worse case for SQL but handled quite elegantly in Solr. Hope this helps! Cheers François On Nov 1, 2011, at 7:46 AM, arshad ansari wrote: Hi, Is SQL Like operator feature available in Apache Solr Just like we have it in SQL. SQL example below - *Select * from Employee where employee_name like '%Solr%'* If not is it a Bug with Solr. If this feature available, please tell the examples available. Thanks! -- Best Regards, Arshad
simple persistance layer on top of Solr
Greetings guys, I have been thinking of using Solr as a simple database due to it's blinding speed -- actually I've used that approach in some projects with decent success. Any thoughts on that? Thanks, MM.
Re: simple persistance layer on top of Solr
Well I want something beyond a key value store. I want to be able to free-text search documents I want to be able to retrieve documents based on other criteria I'm not sure how that would compare with something like MongoDB. Thanks. On Tue, Nov 1, 2011 at 11:49 AM, Walter Underwood wun...@wunderwood.orgwrote: Other than it isn't a database? If you want a key/value store, use one of those. If you want a full DB with transactions, use one of those. wunder On Nov 1, 2011, at 8:47 AM, Memory Makers wrote: Greetings guys, I have been thinking of using Solr as a simple database due to it's blinding speed -- actually I've used that approach in some projects with decent success. Any thoughts on that? Thanks, MM.
Re: simple persistance layer on top of Solr
Well, I've done a lot of work with MySQL and content management systems -- and frankly whenever I have to integrate with Solr or do some Lucene work I am amazed at the speed -- even when I index web pages for search -- MySQL pales by comparison when data sets get large (2 million rows) Thanks, MM. On Tue, Nov 1, 2011 at 12:01 PM, Robert Stewart bstewart...@gmail.comwrote: One other potentially huge consideration is how updatable you need documents to be. Lucene only can replace existing documents, it cannot modify existing documents directly (so an update is essentially a delete followed by an insert of a new document with the same primary key). There are performance considerations here as well (how to do bulk updates quickly, etc.). Bob On Nov 1, 2011, at 11:47 AM, Memory Makers wrote: Greetings guys, I have been thinking of using Solr as a simple database due to it's blinding speed -- actually I've used that approach in some projects with decent success. Any thoughts on that? Thanks, MM.
Re: Is there a good web front end application / interface for solr
Looks very interesting -- actually I looked at it a while back but in a different context -- for a non RoR person how much of a learning curve is it to set up? Thanks. On Tue, Oct 25, 2011 at 5:49 AM, Erik Hatcher erik.hatc...@gmail.comwrote: Blacklight - http://projectblacklight.org/ It's a full featured application fronting Solr. It's Ruby on Rails based, and powers many library front-ends but is becoming much more general purpose for other domains. See examples here: https://github.com/projectblacklight/blacklight/wiki/Examples Also, the forensics domain has used it as well, as mentioned in the slides and talk I attended at Lucene Revolution earlier this year: http://www.lucidimagination.com/blog/2011/06/01/solr-and-law-enforcement-highly-relevant-results-can-be-a-crime/ Often the decision for an application layer like this is determined by the programming language and frameworks used. Blacklight is opinionated (as any other concrete implementation would be) in this regard. If it fits your tastes, it's a great technology to use. Erik On Oct 24, 2011, at 15:56 , Memory Makers wrote: Greetings guys, Is there a good front end application / interface for solr? Features I'm looking for are: configure query interface (using non programatic features) configure pagination configure bookmarking of results export results of a query to a csv or other format (JSON, etc.) Is there any demand for such an application? Thanks.
Re: Is there a good web front end application / interface for solr
Kool -- I was hoping to avoid adding another language :-( python/java/php were going to be it for me -- but I guess not. Thanks. On Tue, Oct 25, 2011 at 6:02 AM, Erik Hatcher erik.hatc...@gmail.comwrote: You could be up and running with Blacklight by following the quickstart instructions in only a few minutes, but Ruby and RoR know-how will be needed to go further with the types of customizations you mentioned. Some things will be purely in configuration sections (but still within Ruby code files) and done easily, but some other customizations will require deeper knowledge. With only a few minutes (given the prerequisites already installed) to give it a try, might as well give it a go :) The Blacklight community is very helpful too, so ask on their e-mail list for assistance, or tap into the #blacklight IRC channel. Erik On Oct 25, 2011, at 05:53 , Memory Makers wrote: Looks very interesting -- actually I looked at it a while back but in a different context -- for a non RoR person how much of a learning curve is it to set up? Thanks. On Tue, Oct 25, 2011 at 5:49 AM, Erik Hatcher erik.hatc...@gmail.com wrote: Blacklight - http://projectblacklight.org/ It's a full featured application fronting Solr. It's Ruby on Rails based, and powers many library front-ends but is becoming much more general purpose for other domains. See examples here: https://github.com/projectblacklight/blacklight/wiki/Examples Also, the forensics domain has used it as well, as mentioned in the slides and talk I attended at Lucene Revolution earlier this year: http://www.lucidimagination.com/blog/2011/06/01/solr-and-law-enforcement-highly-relevant-results-can-be-a-crime/ Often the decision for an application layer like this is determined by the programming language and frameworks used. Blacklight is opinionated (as any other concrete implementation would be) in this regard. If it fits your tastes, it's a great technology to use. Erik On Oct 24, 2011, at 15:56 , Memory Makers wrote: Greetings guys, Is there a good front end application / interface for solr? Features I'm looking for are: configure query interface (using non programatic features) configure pagination configure bookmarking of results export results of a query to a csv or other format (JSON, etc.) Is there any demand for such an application? Thanks.
Re: Is there a good web front end application / interface for solr
Well https://github.com/evolvingweb/ajax-solr is fairly decent for that -- haven't used it in a while but that is a minimalist client -- however I find it hard to customize. MM. On Tue, Oct 25, 2011 at 8:34 AM, Fred Zimmerman zimzaz@gmail.comwrote: what about something that's a bit less discovery-oriented? for my particular application I am most concerned with bringing back a straightforward top ten answer set and having users look at it. I actually don't want to bother them with faceting, etc. at this juncture. Fred On Tue, Oct 25, 2011 at 7:40 AM, Erik Hatcher erik.hatc...@gmail.com wrote: On Oct 25, 2011, at 07:24 , Robert Stewart wrote: It is really not very difficult to build a decent web front-end to SOLR using one of the available client libraries Or even just not using any client library at all (other than an HTTP library). I've done a bit of proof-of-concept/prototyping with a super light weight (and of course Ruby!) approach with my Prism tinkering: https://github.com/lucidimagination/Prism Yes, in general it's very straightforward to build a search UI that shows results, pages through them, displays facets, and allows them to be clicked and filter results and so on. Devil is always in the details, and having saved searches, export, customizability, authentication, and so on makes it a more involved proposition. If you're in a PHP environment, there is VUFind... again pretty library-centric at first, but likely flexible enough to handle any Solr setup - http://vufind.org/. For the Pythonistas, there's Kochief - http://code.google.com/p/kochief/ Being a Rubyist myself (and founder of Blacklight), I'm not intimately familiar with the other solutions but the library world has done a lot to get this sort of thing off the ground in many environments. Erik
Points to processing hastags
Greetings, I am trying to index hashtags from twitter -- so they are tokens that start with a # symbol and can have any number of alpha numeric characters. Examples: 1. #jane 2. #Jane 3. #Jane! At a high level I'd like to be able to: 1. differentiate between say #jane and #jane! 2. differentiate between a hashtag such as #jane and a regular text token jane 3. ask for variation on #jane -- by this I mean #jane? #jane!!! #jane!?!?? are all variations of jane I'd appreciate points to what my considerations should be when I attempt to do the above. Thanks, MM.
Re: questions on query format
Thanks, ?q.alt=*:* worked for me -- how do I make sure that the standard query parser is configured. Thanks. MM. On Mon, Oct 24, 2011 at 2:47 AM, Ahmet Arslan iori...@yahoo.com wrote: 2. If send solr the following query: q=*:* I get nothing just: responseresult name=response numFound=0 start=0 maxScore=0.0/lst name=highlighting//response Would appreciate some insight into what is going on. If you are using dismax as query parser, then *:* won't function as match all docs query. To retrieve all docs - with dismax - use q.alt=*:* parameter. Also, adding debugQuery=on will display information about parsed query.
Is there a good web front end application / interface for solr
Greetings guys, Is there a good front end application / interface for solr? Features I'm looking for are: configure query interface (using non programatic features) configure pagination configure bookmarking of results export results of a query to a csv or other format (JSON, etc.) Is there any demand for such an application? Thanks.
questions on query format
Hi,
I've spent quite some time reading up on the query format and can't seem to
solve this problem:
1. If send solr the following query:
q={!lucene}profile_description:*
I get what I would expect.
2. If send solr the following query:
q=*:*
I get nothing just:
responseresult name=response numFound=0 start=0
maxScore=0.0/lst name=highlighting//response
Would appreciate some insight into what is going on.
Thanks.
