Re: Faceting return value of a function query?

2014-11-03 Thread Tom
I see. Thank you! :-) Sent from my Android phone On Nov 3, 2014 9:35 PM, "Erick Erickson" wrote: > Yep. It's almost always easier and faster if you can pre-compute as > much as possible during indexing time. It'll take longer to index of > course, but the ratio of writing to the index to searc

Re: Data colocation hint to solr index

2014-11-03 Thread Shawn Heisey
On 11/3/2014 12:45 PM, maninder batth wrote: > Thank you for recommendation on composite IDs. We currently use solr 3.x. > After reading on composite ids, it sounds like a feature of solr 4.x. Is > something similar available in solr 3.x also? Also, we do not use solrCloud. The compositeId router

Admin UI Schema Browser screen and ReverseStringFilterFactory

2014-11-03 Thread Alexandre Rafalovitch
Hello, I just noticed this one and curious what's causing this (desirable I guess) behavior. I have a chain with ReverseStringFilterFactory in it (both index and query). So, the tokens are reversed from the input. But when I look at the Schema Browser screen and it loads the tokens, it seems to

Re: Faceting return value of a function query?

2014-11-03 Thread Erick Erickson
Yep. It's almost always easier and faster if you can pre-compute as much as possible during indexing time. It'll take longer to index of course, but the ratio of writing to the index to searching is usually hugely in favor of doing the work during indexing. Best, Erick On Mon, Nov 3, 2014 at 8:

Re: Missing log entries with log4j log rotation

2014-11-03 Thread Shawn Heisey
On 11/1/2014 11:45 AM, Shawn Heisey wrote: > There appear to be large blocks of time missing in my solr logfiles > created with slf4j->log4j and rotated using the log4j config: > > End of solr.log.1: INFO - 2014-10-31 12:52:25.073; > Start of solr.log: INFO - 2014-11-01 02:27:27.404; > > End of

Re: Faceting return value of a function query?

2014-11-03 Thread Tom
Hi Erik, Thanks for the reply! Do you mean parse and modify the documents before sending them to Solr? Cheers, Yubing On Mon, Nov 3, 2014 at 8:48 PM, Erick Erickson wrote: > Wouldn't it be easiest to compute the span at index time? Then it's > very straight-forward. > > Best, > Erick > > On Mo

Re: Faceting return value of a function query?

2014-11-03 Thread Erick Erickson
Wouldn't it be easiest to compute the span at index time? Then it's very straight-forward. Best, Erick On Mon, Nov 3, 2014 at 8:18 PM, Yubing (Tom) Dong 董玉冰 wrote: > Hi, > > I'm new to Solr, and I'm having a problem with faceting. I would really > appreciate it if you could help :) > > I have a

Faceting return value of a function query?

2014-11-03 Thread Tom
Hi, I'm new to Solr, and I'm having a problem with faceting. I would really appreciate it if you could help :) I have a set of documents in JSON format, which I could post to my Solr core using the post.jar tool. Each document contains two fields, namely "startDate" and "endDate", both of which a

Re: Solr error : sorry, no dataimport-handler defined!

2014-11-03 Thread Tim Dunphy
Hi Alexandre, OK some good progress was made based on this advice. Thanks! I think we're in the home stretch with the data import. Not there yet. But hopefully close. > Two problems: > 1) You have (span) elements in your solrconfig.xml. They just > do not belong there. The original tutorial sc

RE: Missing Records

2014-11-03 Thread AJ Lemke
So I jumped back on this. I have not been using the optimize option on this new set of tests. If I run the full index on the leader I seem to get all of the items in the database minus 3 that have a missing field. Indexing completed. Added/Updated: 903,990 documents. Deleted 0 documents. (Durat

Re: Data colocation hint to solr index

2014-11-03 Thread Erick Erickson
You have a TB-scale index and you're not using SolrCloud? Are you using master/slave or otherwise splitting up your index? Because if you're not, then please ship me some of your hardware because it must be awesome. Which is a tongue-in-cheek way of saying there must be lots of details you aren't

Re: Consul instead of ZooKeeper anyone?

2014-11-03 Thread Erick Erickson
bq: Do you think it would be possible to add an abstraction layer to Solr source code in near future? I strongly doubt it. As you've already noted, this is a large amount of work. Without some super-compelling advantage I just don't see the interest. bq: to avoid deploying ZK just for SolrCloud

custom sorting of search result

2014-11-03 Thread alxsss
Hello, We need to order solr search results according to specific rules. I will explain with an example. Let say solr returns 1000 results for query "sport". These results must be divided into three buckets according to rules that come from database. Then one doc must be chosen from each b

Re: Which Solr releases contain SOLR-4470 (Security for inter-solr-node requests)

2014-11-03 Thread Chris Hostetter
: I am currently working on SolrCloud and its related security : configurations for securing Solr web applications using HTTP Basic : Authentication mechanism. Among the Solr nodes inside the SolrCloud : clustered env, there seem to be existing some inter-solr-node : communication issues due t

Re: Which Solr releases contain SOLR-4470 (Security for inter-solr-node requests)

2014-11-03 Thread Jan Høydahl
You find the answer to such questions by looking at the state of the JIRA issue https://issues.apache.org/jira/browse/SOLR-4470 Staus: Open Fix version: Trunk Which means that this feature is not included in any released Solr version (yet). -- Jan Høydahl, search solution architect Cominvent AS

Which Solr releases contain SOLR-4470 (Security for inter-solr-node requests)

2014-11-03 Thread Yuan Jerry
I am currently working on SolrCloud and its related security configurations for securing Solr web applications using HTTP Basic Authentication mechanism. Among the Solr nodes inside the SolrCloud clustered env, there seem to be existing some inter-solr-node communication issues due to the securi

Re: Data colocation hint to solr index

2014-11-03 Thread maninder batth
Thank you for recommendation on composite IDs. We currently use solr 3.x. After reading on composite ids, it sounds like a feature of solr 4.x. Is something similar available in solr 3.x also? Also, we do not use solrCloud. On Mon, Nov 3, 2014 at 12:41 PM, Shalin Shekhar Mangar < shalinman...@gmai

Re: Cannot use Phrase Queries in eDisMax and filtering

2014-11-03 Thread Tim Hearn
It is of type string. On Mon, Nov 3, 2014 at 2:29 PM, Ramzi Alqrainy wrote: > I tried to produce your case in my machine with below queries, but > everything > worked fine with me. I just want to ask you a question what is the field > type of "tag" field ? > > q=bmw& > fl=score,*& > wt=json& > f

Re: Cannot use Phrase Queries in eDisMax and filtering

2014-11-03 Thread Ramzi Alqrainy
I tried to produce your case in my machine with below queries, but everything worked fine with me. I just want to ask you a question what is the field type of "tag" field ? q=bmw& fl=score,*& wt=json& fq=city_id:59& qt=/query& defType=edismax& pf=title^15%20discription^5& pf3=1& pf2=1& ps=1& qroup

Re: Consul instead of ZooKeeper anyone?

2014-11-03 Thread Greg Solovyev
Thanks Erick, after looking further into Solr's source code, I see that it's married to ZK libraries and it won't be possible to extend existing code without diverting from the trunk. At the same time, I don't see any reason for lack of abstraction in cloud-related code of Solr and SolrJ. As fa

Re: Cannot use Phrase Queries in eDisMax and filtering

2014-11-03 Thread Tim Hearn
That was a typo in the email I did not actually send the query with a start param of 50. I sent it with a start param of 0, I just verified. Sorry for the mistake. On Mon, Nov 3, 2014 at 1:41 PM, Ramzi Alqrainy wrote: > The results are different, because you need to set "start" parameter 0 > i

Re: Solr error : sorry, no dataimport-handler defined!

2014-11-03 Thread Alexandre Rafalovitch
Two problems: 1) You have (span) elements in your solrconfig.xml. They just do not belong there. The original tutorial screwed up. Your element should be on the same level as the other elements in that example. 2) You also seem to have another random piece of data configuration in the solrconfig.x

Re: Cannot use Phrase Queries in eDisMax and filtering

2014-11-03 Thread Ramzi Alqrainy
The results are different, because you need to set "start" parameter 0 instead of 50 in the first query (after filtration ) with same rows value -- View this message in context: http://lucene.472066.n3.nabble.com/Cannot-use-Phrase-Queries-in-eDisMax-and-filtering-tp4167302p4167329.html Sent fr

Re: Solr error : sorry, no dataimport-handler defined!

2014-11-03 Thread Tim Dunphy
Hi Alexandre, Thanks so much for your input and examples! Ok so here's what I've done so far with no luck as of yet unfortunately. Inside of solrconfig.xml I put the following: ** As you can see, I've replaced the relative paths with absolute ones. So as of now, m

Re: Solr slow start up (tlog is small)

2014-11-03 Thread Po-Yu Chuang
Hi Yonik, After removing the suggest component, it takes only 7 seconds to start up now!!! Thank you so much. Po-Yu On Mon, Nov 3, 2014 at 11:17 AM, Yonik Seeley wrote: > Can you tell from the logs what Solr is doing during that time? > Do you have any warming queries configured? > Also see th

Re: Data colocation hint to solr index

2014-11-03 Thread Shalin Shekhar Mangar
If you're using SolrCloud then you can use composite IDs such as !doc-id to co-locate documents belonging to a manufacturer together and at query time, you can add _route_=! to the request to route it to the correct node. On Mon, Nov 3, 2014 at 11:00 PM, maninder batth wrote: > Hi, > In my compa

Data colocation hint to solr index

2014-11-03 Thread maninder batth
Hi, In my company, we serve car manuals for different car manufacturers with their various makes and models. Typically, the search is always done within context of a car manufacturer, year, make and model. Is there a way in Solr to create indexes based on this criteria? Currently, the index contain

Re: Solr slow start up (tlog is small)

2014-11-03 Thread Dmitry Kan
One other reason for a slow start-up can be large number of segments in the index. Which I'm guessing is not the case since you optimized? But anyway, what's the number of segments in both 280G and 185G indices? Dmitry On Mon, Nov 3, 2014 at 6:17 PM, Yonik Seeley wrote: > Can you tell from the

Cannot use Phrase Queries in eDisMax and filtering

2014-11-03 Thread Tim Hearn
I am writing a search bar application with Solr which I'd like to have the following two features: phrase matching for user queries - results which match user phrase are boosted. Field faceting based on 'tags' field. When I execute this query: q=steve jobs& fq=storeid:527bd613e4b0564cc755460a&

Re: order of updates

2014-11-03 Thread Matteo Grolla
Thanks really a lot Yonik! Il giorno 03/nov/2014, alle ore 15:51, Yonik Seeley ha scritto: > On Mon, Nov 3, 2014 at 8:53 AM, Matteo Grolla wrote: >> HI, >>can anybody give me a confirm? >> If I add multiple document with the same id but differing on other fields >> and then issue a comm

Re: Solr slow startup

2014-11-03 Thread Michal Krajňanský
Hey Yonik, That (getting rid of the suggester) solved the issue! You saved me a lot of time and nerves. Best, Michal 2014-11-03 17:19 GMT+01:00 Yonik Seeley : > One possible cause of a slow startup with the default configs: > https://issues.apache.org/jira/browse/SOLR-6679 > > -Yonik > http://

Re: Solr slow startup

2014-11-03 Thread Yonik Seeley
One possible cause of a slow startup with the default configs: https://issues.apache.org/jira/browse/SOLR-6679 -Yonik http://heliosearch.org - native code faceting, facet functions, sub-facets, off-heap data On Mon, Nov 3, 2014 at 11:05 AM, Michal Krajňanský wrote: > Dear All, > > > Sorry for t

Re: Solr slow start up (tlog is small)

2014-11-03 Thread Yonik Seeley
Can you tell from the logs what Solr is doing during that time? Do you have any warming queries configured? Also see this: https://issues.apache.org/jira/browse/SOLR-6679 (comment out suggester related stuff if you aren't using it) -Yonik http://heliosearch.org - native code faceting, facet func

Solr slow startup

2014-11-03 Thread Michal Krajňanský
Dear All, Sorry for the possibly newbie question as I have only recently started experimenting with Solr and Solrcloud. I am trying to import an index originally created with Lucene 2.x so Solr 4.10. What I did was: 1. upgrade index to version 3.x with IndexUpgrader 2. upgrade index to version

Solr slow start up (tlog is small)

2014-11-03 Thread Po-Yu Chuang
Hi, I am using Solr 4.9 with Tomcat and it works fine except that the deployment of solr.war is too long. While deploying Solr, all webapps on Tomcat stop responding which is unacceptable. Most articles I found say that it might result from big transaction log because of uncommitted documents, but

Re: SOLRJ - query with ChildDocTransformerFactory crash because of the javabin parser

2014-11-03 Thread Erick Erickson
bq: Was a little, let's say mistake, in my query Been there, done that ;) Thanks for closing this out. Best, Erick On Mon, Nov 3, 2014 at 3:25 AM, andreic9203 wrote: > Hello, > > And to answer to my question...:D > > Was a little, let's say mistake, in my query. > Instead of > fl=[child parentF

Re: Ignoring Duplicates in Multivalue Field

2014-11-03 Thread Matthew Nigl
>From memory, if you use UniqFieldsUpdateProcessor after DistributedUpdateProcessor, then you will be filtering on the set ["1", "2", "3", "2"]. ** * * * * * myMultValueField * * * ** **

Re: order of updates

2014-11-03 Thread Yonik Seeley
On Mon, Nov 3, 2014 at 8:53 AM, Matteo Grolla wrote: > HI, > can anybody give me a confirm? > If I add multiple document with the same id but differing on other fields and > then issue a commit (no commits before this) the last added document gets > indexed, right? Correct. > using sol

Re: dynamically change default update chain

2014-11-03 Thread Dmitry Kan
An update: Another idea comes from Erick Hatcher; sharing it for the benefit of anyone who's interested in the topic: maybe you can make a custom request handler that toggles which is the default chain? On Mon, Nov 3, 2014 at 4:08 PM, Dmitry Kan wrote: > Thanks, Mike, > > we have discussed

Re: Ignoring Duplicates in Multivalue Field

2014-11-03 Thread Jack Krupansky
The update processors are only processing the values in the "source" data, not the data that has already been indexed and stored. We probably need to file a Jira to add an "insert" field value option that merges in the new field value, skipping it if it already exists or appending it to the en

Re: SolrCloud use of "min_rf" through SolrJ

2014-11-03 Thread Zisis Tachtsidis
In case anyone else runs into this, I've managed to make it work. I didn't notice in the ticket discussion that the specific feature is enabled when min_rf >=2, I was setting min_rf=1. It goes without saying that you should also have at least 2 replicas in your SolrCloud configuration. The actual c

Re: dynamically change default update chain

2014-11-03 Thread Dmitry Kan
Thanks, Mike, we have discussed something similar with steffkes on IRC today, where I said: "some programmatic convenience would be great of course. But I could in principle imagine having two versions of solrconfig.xml and swapping them followed by a core reload. It just sounds a bit scary to me.

order of updates

2014-11-03 Thread Matteo Grolla
HI, can anybody give me a confirm? If I add multiple document with the same id but differing on other fields and then issue a commit (no commits before this) the last added document gets indexed, right? H.p. using solr 4 and default settings for optimistic locking. Matteo

RE: FOSDEM 2015 - Open Source Search Dev Room

2014-11-03 Thread Uwe Schindler
Hi, forgot to mention: FOSDEM 2015 takes place in Brussels on January 31th and February 1st, 2015. See also: https://fosdem.org/2015/ I hope to see you there! Uwe > -Original Message- > From: Uwe Schindler [mailto:uschind...@apache.org] > Sent: Monday, November 03, 2014 1:29 PM > To: d.

Re: dynamically change default update chain

2014-11-03 Thread Michael Sokolov
Just to get the obvious sledgehammer solution out of the way - upload a new, edited solrconfig.xml with the default changed, and reload the core. -Mike On 11/3/14 6:28 AM, Dmitry Kan wrote: Hello solr fellows, I'm working on a project that involves using two update chains. One default chain

CFP: FOSDEM 2015 - Open Source Search Dev Room

2014-11-03 Thread Uwe Schindler
***Please forward this CFP to anyone who may be interested in participating.*** Hi, Search has evolved to be much more than simply full-text search. We now rely on “search engines” for a wide variety of functionality: search as navigation, search as analytics and backend for data visualization

dynamically change default update chain

2014-11-03 Thread Dmitry Kan
Hello solr fellows, I'm working on a project that involves using two update chains. One default chain is used most of the time and another one custom is used sporadically. The default update chain is called automatically without action needed (well, that's why it is default). The custom pipeline

Re: SOLRJ - query with ChildDocTransformerFactory crash because of the javabin parser

2014-11-03 Thread andreic9203
Hello, And to answer to my question...:D Was a little, let's say mistake, in my query. Instead of fl=[child parentFilter="cat:PARENT" childFilter="cat:CHILD"] should be fl=id,[child parentFilter="cat:PARENT" childFilter="cat:CHILD"] Awkward, because if you put the query and the filter in URL or

RE: Ignoring Duplicates in Multivalue Field

2014-11-03 Thread Tomer Levi
Hi Ahmet, When I add the RunUpdateProcessorFactory Solr didn't remove any duplications. Any other idea? -Original Message- From: Ahmet Arslan [mailto:iori...@yahoo.com.INVALID] Sent: Monday, November 03, 2014 1:35 AM To: solr-user@lucene.apache.org Subject: Re: Ignoring Duplicates in Mul

AW: Spellchecking and suggesting part numbers

2014-11-03 Thread Lochschmied, Alexander
Thanks James, this did help a lot. Is it possible to make DirectSolrSpellChecker try to return suggestions with maximum length of matching leading characters? Alexander -Ursprüngliche Nachricht- Von: Dyer, James [mailto:james.d...@ingramcontent.com] Gesendet: Mittwoch, 24. September 20