Re: Core admin: create new core
Solr 5 has been released. I was just giving a try and come across the same issue. As I heard over from some documentation, Solr 5 doesn't come with default core (example in earlier versions). And this requires us to create a core from Solr Admin. When I tried to create the core, I get the following error Error CREATEing SolrCore 'inventory': Unable to create core [inventory] Caused by: Can't find resource 'solrconfig.xml' in classpath or '/Users/manjunath.reddy/Programming/Solr/solr-5.0.0/server/solr/inventory/conf' So I had to manually create the core based on my previous experience with Solr 4.10 version. I guess its quite misleading for new users of Solr. I like the older versions of Solr that comes with default cores so that It would be easier to follow up on. I have attached screen shots for the reference. Is there a work around for this? http://lucene.472066.n3.nabble.com/file/n4191378/solr-1.png http://lucene.472066.n3.nabble.com/file/n4191378/solr-2.png -- View this message in context: http://lucene.472066.n3.nabble.com/Core-admin-create-new-core-tp4099127p4191378.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Applying boosting for keyword search
Hi Jack, Thank you for the suggestions. :-) -- View this message in context: http://lucene.472066.n3.nabble.com/Applying-boosting-for-keyword-search-tp4137523p4138239.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Applying boosting for keyword search
Hi Erick, Your explanation leads me to one question :-) if */select?q=featured:true^100fq=make:toyotasort=featured_date desc,price asc* The above query, without edismax, works well because, If I'm not mistaken its boosting document by value method. So I'm boosting all my documents with the value featured=true and all those documents would be sorted by their featured date in descending order (Latest featured documents) and price (lower to higher). My question is, If we were to boost the documents based on a value, how could we make sure the order of the documents? For example : https://wiki.apache.org/solr/SolrRelevancyFAQ defType=dismaxqf=textq=supervilliansbf=popularity In the above case, all the documents that contains the word *popularity* would be on top depends on their score. However, I want to order the documents by certain criteria that contains the word popularity So we would have to use *sort* to order the documents. if we say, boosting has no or almost no effect if we use sort, then whats the contradiction story between *sort* and *boost* :-) would be interesting to know the answer -- View this message in context: http://lucene.472066.n3.nabble.com/Applying-boosting-for-keyword-search-tp4137523p4138241.html Sent from the Solr - User mailing list archive at Nabble.com.
Applying boosting for keyword search
HI, I have a scenario where by I apply boosting in the following two cases - Usual search, by user selection - Keyword search. I have a field *keyword* that is copy/combination of many fields When user does the usual query, my boosting works fines, this is how I do boosting /select?q=featured:true^100fq=make:toyotasort=price asc This works fine However I'm confused, how we could perform boosting using *keyword* serach /select?q=toyotadefType=edismaxbq=featured:true^100sort=price asc This does not work as I expected. In solrconfig.xml my default field set to keyword (df=keyword) My question, how could I perform boosting with keyword search? when we are using keyword search, user keys goes into *q* param, so where Can I specify the boosting query? I tried with *bq* and *function query*, No luck :-(. Can I perform like this /select?q=toyota featured:true^100 Keyword and boosting done with *q* param. -- View this message in context: http://lucene.472066.n3.nabble.com/Applying-boosting-for-keyword-search-tp4137523.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Using fq as OR
The *fq* is used for searching more deterministic results something like
WHERE type={}
Where as *q* is something like WHERE type like '%%'
user *fq*, if your are sure of what your going to search
use *q*, if not sure what your trying to search
If you are using fq and if you do not get any matching documents, solr
throws 0 or error message
where q would try to match nearest documents for your search query
That's what I have experienced so far. :-).
--
View this message in context:
http://lucene.472066.n3.nabble.com/Using-fq-as-OR-tp4137411p4137525.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Applying boosting for keyword search
Hi Jack, Thanks for your help. I do not want to boost *keyword* field. I apply full text search no keyword field and boost based on another field *featured*. Also qf field allows us to boost the field without values. I would like to boost with value Ex: qf=featured:true^100 - I don't think this is correct Example : Get all Toyota 2013 Car listings, which are featured listings. Field *featured* is a boolean flag So I tried this way /select?q=toyota 2014defType=edismaxq.op=ANDqf=featured:true^100wt=jsonsort=price asc No luck :-(. My boosting works fine without keyword, I only have issue having keyword search. -- View this message in context: http://lucene.472066.n3.nabble.com/Applying-boosting-for-keyword-search-tp4137523p4137528.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Sharing configuration files among cores [4.7.1]
Thanks Dmitry, Yes I'm on *nix OS. Yes *soft-link* was mentioned by one of my sys-admin friend as well. :-). I was just trying to from the community if there there would be a way from Solr itself. Thanks for suggestion. Probably soft-link is the way to go now :-). -- View this message in context: http://lucene.472066.n3.nabble.com/Sharing-configuration-files-among-cores-4-7-1-tp4130928p4130997.html Sent from the Solr - User mailing list archive at Nabble.com.
Sharing configuration files among cores [4.7.1]
Hi All, I'm using latest version of Solr 4.7.1 and was wondering if there is a way to share common schema.xml. Here is my use case. I have two cores which would have same schema structure. Core 1 = Stores Active Records Core 2 = All records regardless of the status Both the cores would have the same schema structure however data-config.xml (DIH) would be different as the query would change based on the record status. As of now, I created two cores *Core 1* - conf -- data-config.xml -- schema.xml -- solrconfig.xml - data - core.properties *Core 2* - conf -- data-config.xml -- schema.xml -- solrconfig.xml - data - core.properties In such scenario, is there a way to share same schema.xml file between two cores :-). -- View this message in context: http://lucene.472066.n3.nabble.com/Sharing-configuration-files-among-cores-4-7-1-tp4130928.html Sent from the Solr - User mailing list archive at Nabble.com.
Does CachedSqlEntityProcessor works?
I tried to use *CachedSqlEntityProcessor* in DataImportHandler with
Sub-entity query. It does not seems to be working.
Here is my query
entity name=listing dataSource=mysql query=SELECT id,make, model FROM
LISTING
entity name=account dataSource=mssql query=SELECT name,email FROM
CUSTOMER WHERE id='${listing.customer_id}'
processor=CachedSqlEntityProcessor
/entity
/entity
If I remove *processor=CachedSqlEntityProcessor*, run without
CachedSqlEntityProcessor it works fine. With CachedSqlEntityProcessor my
sub-entity some how fails to load the data?
My Solr version 4.3.1
I also tried to use other options of CachedSqlEntityProcessor
http://wiki.apache.org/solr/DataImportHandler#CachedSqlEntityProcessor
Is CachedSqlEntityProcessor deprecated in 4.3.1?
--
View this message in context:
http://lucene.472066.n3.nabble.com/Does-CachedSqlEntityProcessor-works-tp4124772.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: start.jar config
Solr already writes the logs to a file 'solr.log'. Its located in the same folder as start.jar (logs/solr.log). I'm not sure if thats what you looking for :-). -- View this message in context: http://lucene.472066.n3.nabble.com/start-jar-config-tp4119201p4119203.html Sent from the Solr - User mailing list archive at Nabble.com.
URLDataSource : Issue assigning single xpath field name to two solr fields
I'm not sure if I would be missing any configuration params here, however when I tried to assign an xpath field from URLDataSource (XML end point) to two fields defined in schema.xml. Here is my scenario, I have two fields *profile_display* and *profile_indexed* My assignment in DataImpotHandler looks like this entity name=user dataSource=crm url=http://URLTOExternalSystem//ProfileService.svc/; processor=XPathEntityProcessor forEach=/Profiles field column=profile_display xpath=/Profile/profile_name / field column=profile_indexed xpath=/Profile/profile_name / /entity My Scheama.xml config looks like this field name=profile_display type=string indexed=true stored=true multiValued=false default=/ field name=profile_indexed type=text_general indexed=true stored=false multiValued=false default=/ *So the issue here is, the value is value is always assigned to profile_indexed, and profile_display does not contain any value. * Meaning, if we were to assign xpath field name to different solr fields, only the last field contains the data. The reason I have two fields is that, One to store it as a String to display to user, another field where I apply Filter and Tokenizers to do text transformation. -- View this message in context: http://lucene.472066.n3.nabble.com/URLDataSource-Issue-assigning-single-xpath-field-name-to-two-solr-fields-tp4119206.html Sent from the Solr - User mailing list archive at Nabble.com.
Issue with PHP urlencode and solr encoding
Hi,
I come across the issue with urlencoding between PHP and Solr.
I have a field indexed with value *WBE(Honda Edix)* in Solr.
From PHP codes, if I urlencode($string) and send to Solr, I do not get the
accurate results.
Here is the part of the solr query *fq=model:WBE(Honda+Edix)*
However, If I do it *fq=model:WBE\(Honda+Edix\)* this way directly from
Solr, I would get the accurate results.
I assume that the '(' and ')' part of the solr query.
How do I escape '(' and ')' from the client side.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Issue-with-PHP-urlencode-and-solr-encoding-tp4119176.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Issue with PHP urlencode and solr encoding
Hi Shawn and Rico,
Thanks you for your suggestions, those are valuable suggestions :-).
If Pharse Query does not work as we expected sometimes, I guess we could use
*TermQuery* instead.
http://blog.florian-hopf.de/2013/01/make-your-filters-match-faceting-in-solr.html
This worked fine *fq={!term%20f=model%20v=WBE(Honda%20Edix)}*.
I agree with Shawn comments that Escaping query characters must be done
before URL encoding.
:-).
Thanks again for your replies.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Issue-with-PHP-urlencode-and-solr-encoding-tp4119176p4119187.html
Sent from the Solr - User mailing list archive at Nabble.com.
Indexing strategies?
Hi, I'm facing a dilemma of choosing the indexing strategies. My application architecture is - I have a listing table in my DB - For each listing, I have 3 calls to a URL Datasource of different system I have 200k records Time taken to index 25 docs is 1Minute, so for 200k it might take more than 100hrs :-(? I know there are lot of factors to consider from Network to DB. I'm looking for different strategies that we could perform index. - Can we run multiple data import handlers? one data-config for first 100k and second one is for another 100k - Would it be possible to write java service using SolrJ and perform multi-threaded calls to Solr to Index? - The URL Datasources i'm using is actually resided in MSSQL database of different system. Could I be able to fasten indexing time if I just could use JDBCDataSource that calls DB directly instead through API URL data source? Is there any other strategies we could use? Thank you, -- View this message in context: http://lucene.472066.n3.nabble.com/Indexing-strategies-tp4116852.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Indexing strategies?
Hi Erick, Thank you very much, those are valuable suggestions :-). I would give a try. Appreciate your time. -- View this message in context: http://lucene.472066.n3.nabble.com/Indexing-strategies-tp4116852p4117050.html Sent from the Solr - User mailing list archive at Nabble.com.
Difference in boosting with q and eDisMax. Are they same or different?
Hi, In an application I'm working on, I have applied two types of search criteria to Solr. First one is a definitive search using filter query (fq) where by I know exactly the fields I'm searching for. Secondly I use eDisMax for full-text search, this would be applied when user enters keywords. The keyword field is combination of fields (using copyField). Is there a difference in applying boosting to 'q' query and eDisMax query? If I apply boosting to q, the way I do it is select/?q=listing_type:bump^100 listing_type:featured^10 (*:*)^1bq=recip(rord(creation_date),1,3,3)^1.5 If I'm using eDisMax what is the appropriate way of using boosting? I'm bit confused with using params when we apply eDisMax http://wiki.apache.org/solr/ExtendedDisMax Can I apply boosting query to *q.alt* Ex: select/?defType=edisMaxq=*:*q.alt=listing_type:bump^100 listing_type:featured^10 (*:*)^1bq=recip(rord(creation_date),1,3,3)^1.5 what is the significance of qf,pf,pf2, bq, bf and boost in eDisMax for boosting documents? -- View this message in context: http://lucene.472066.n3.nabble.com/Difference-in-boosting-with-q-and-eDisMax-Are-they-same-or-different-tp4116418.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Storing MYSQL DATETIME field in solr as String
Hi Tariq, I'm glad that helped you :-). Thanks -- View this message in context: http://lucene.472066.n3.nabble.com/Solved-Storing-MYSQL-DATETIME-field-in-solr-as-String-tp4106836p4112979.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Storing MYSQL DATETIME field in solr as String
I found a way to store MySQL DateTime as a string in Solr Here is the way in data-config.xml in the SQL query we could convert the date directly to char CAST(l.creation_date as char) as creation_date, CAST(l.modification_date as char) as modification_date, in schema.xml field name=creation_date type=string indexed=true stored=true multiValued=false default= / field name=modification_date type=string indexed=true stored=true multiValued=false default= / Output would be str name=creation_date2013-11-13 10:26:32/str str name=modification_date2013-11-13 10:26:32/str This is exactly what I was looking for. If you guys have any other wise please free to share. :-). Happy Solr!!! -- View this message in context: http://lucene.472066.n3.nabble.com/Storing-MYSQL-DATETIME-field-in-solr-as-String-tp4106836p4109720.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Boosting results on value of field different from query
Hi Puneet, http://localhost:8983/solr/my/select?q=type:sedan^100 type:compact^10 (:*)^1wt=jsonindent=truefl=,scoredebug=resultsbf=recip(rord(publish_date),1,2,3)^1.5sort=score desc Not really. The query I mentioned it does not restrict the other types, instead it would push other types to bottom of the list as they have lower score. The query I mentioned, Cars with type sedan listed first, compact secondly and rest are the bottom of the list. You would have to control the logic at the application level, which type you want to be on the top. In the above query, we are boosting type sedan to be on top, compact below the sedan types and rest are at the bottom. Change the those boosting values to get the different results. -- View this message in context: http://lucene.472066.n3.nabble.com/Boosting-results-on-value-of-field-different-from-query-tp4108180p4108636.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Boosting results on value of field different from query
Hi Puneet, if you type field is pre-determined text field ex type [compact, sedan, hatchback], I think you have to boost with query type field (q) to get more accurate boosting. Ex: http://localhost:8983/solr/my/select?q=type:sedan^100 type:compact^10 (:*)^1wt=jsonindent=truefl=,scoredebug=resultsbf=recip(rord(publish_date),1,2,3)^1.5sort=score desc For publish_date, replace with the date you use for getting latest resultes. In the above query, things to note is that - fl=,score - The result set would display score value for each document - sort by score as first sort field that will give you the documents with the highest boost value (score) on top Play around with the boosting values ^100 ^10 (perhaps 5,10,20 ) and observe how the score value will change the documents. I'm not really sure how solr calculation works, however the above query must give you the accurate boosted documents. -- View this message in context: http://lucene.472066.n3.nabble.com/Boosting-results-on-value-of-field-different-from-query-tp4108180p4108190.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Storing MYSQL DATETIME field in solr as String
Hi Raymond, You mean during the DIH execution? -- View this message in context: http://lucene.472066.n3.nabble.com/Storing-MYSQL-DATETIME-field-in-solr-as-String-tp4106836p4107136.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Storing MYSQL DATETIME field in solr as String
Hi Shekhar, Yes :-). Thanks for the references. Will try out the options you mentioned. Thanks -- View this message in context: http://lucene.472066.n3.nabble.com/Storing-MYSQL-DATETIME-field-in-solr-as-String-tp4106836p4107135.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Storing MYSQL DATETIME field in solr as String
Hi Raymond, Let me give a try. Thanks for suggestion :-). Cheers, On Wed, Dec 18, 2013 at 12:38 AM, Raymond Wiker [via Lucene] ml-node+s472066n410713...@n3.nabble.com wrote: On 17 Dec 2013, at 17:30 , manju16832003 [hidden email]http://user/SendEmail.jtp?type=nodenode=4107138i=0 wrote: Hi Raymond, You mean during the DIH execution? Yes; as part of the sql statement. -- If you reply to this email, your message will be added to the discussion below: http://lucene.472066.n3.nabble.com/Storing-MYSQL-DATETIME-field-in-solr-as-String-tp4106836p4107138.html To unsubscribe from Storing MYSQL DATETIME field in solr as String, click herehttp://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=4106836code=bWFuanUxNjgzMjAwM0BnbWFpbC5jb218NDEwNjgzNnwtMTU0ODA3NzA3 . NAMLhttp://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewerid=instant_html%21nabble%3Aemail.namlbase=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespacebreadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml -- Thank you, Manjunath -- View this message in context: http://lucene.472066.n3.nabble.com/Storing-MYSQL-DATETIME-field-in-solr-as-String-tp4106836p4107195.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Storing MYSQL DATETIME field in solr as String
OK here are the scenarios I tried. *Scenario - 1: * dih.xml (aka data-config.xml) entity dataSource=solr name=listing query=... transformer=DateFormatTransformer field name=quot;publish_datequot; column=quot;publish_datequot; xpath=quot;/RDF/item/datequot; dateTimeFormat=quot;lt;b-MM-dd'T'HH:mm:ss* locale=en/ schema.xml field name=publish_date type=date indexed=true stored=true multiValued=false default=NOW / It does not work. Throws an following exception WARN org.apache.solr.handler.dataimport.DateFormatTransformer – Could not parse a Date field java.text.ParseException: Unparseable date: 2013-12-05 15:40:03.0 *Scenario - 2* dih.xml (aka data-config.xml) entity dataSource=solr name=listing query=... transformer=DateFormatTransformer field name=quot;publish_datequot; column=quot;publish_datequot; xpath=quot;/RDF/item/datequot; dateTimeFormat=quot;lt;b-MM-dd HH:mm:ss* locale=en/ schema.xml field name=publish_date type=date indexed=true stored=true multiValued=false default=NOW / Output is date name=publish_date2012-12-05T07:38:27Z/date *Scenario - 3* dih.xml (aka data-config.xml) entity dataSource=solr name=listing query=... transformer=DateFormatTransformer field name=quot;publish_datequot; column=quot;publish_datequot; xpath=quot;/RDF/item/datequot; dateTimeFormat=quot;lt;b-MM-dd HH:mm:ss* locale=en/ schema.xml field name=publish_date type=string indexed=true stored=true multiValued=false default=NOW / Output is str name=publish_dateThu Dec 05 15:38:27 MYT 2013/str My expectation was *2012-12-05 07:38:27* (MySQL DateTime format) -- View this message in context: http://lucene.472066.n3.nabble.com/Storing-MYSQL-DATETIME-field-in-solr-as-String-tp4106836p4106854.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Profiler
Hi NewRelic is good enough to monitor the Solr. Are you using Solarium or SolrJ client to connect to Solr?. We have used Solarium and able to monitor each calls and gather most of the info. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Profiler-tp4106231p4106859.html Sent from the Solr - User mailing list archive at Nabble.com.
Storing MYSQL DATETIME field in solr as String
Hi All, In database, MYSQL DateTime format is 2013-10-17 15:32:48 [-MM-dd hh:mm:ss] however, when I store this DateTime field in solr as String, it would look something like this 2013-10-17 15:32:48*.0*. Why the 0 is appended at the end of the DateTime? For the same date field, I do have another field in solr that stores date as UTC. The reason I'm storing date in String with different field to display user as it appears in MySQL database. But when DateTime stored as String in solr, it appends .0 end of the date. Is it the behaviour of Solr? is there a reason behind that? Thanks -- View this message in context: http://lucene.472066.n3.nabble.com/Storing-MYSQL-DATETIME-field-in-solr-as-String-tp4106836.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Storing MYSQL DATETIME field in solr as String
Hi Shekhar, Thanks for replying :-). Yes your answer is justifiable. DateFormatTransformer was a good consideration. However it doesn't out the format I expected. Rather it outputs in the following manner str name=publish_dateTue Jun 05 00:00:00 MYT 2012/str Here is DIH configuration field name=publish_date column=publish_date xpath=/RDF/item/date dateTimeFormat=-MM-dd'T'HH:mm:ss locale=en/ And also I tried this way field name=publish_date column=publish_date xpath=/RDF/item/date dateTimeFormat=-MM-dd HH:mm:ss locale=en/*-- without having 'T'* Output is the same :-) I expected date format to be *-MM-dd HH:mm:ss* -- View this message in context: http://lucene.472066.n3.nabble.com/Storing-MYSQL-DATETIME-field-in-solr-as-String-tp4106836p4106849.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Storing MYSQL DATETIME field in solr as String
Hi Shekhar, Yes, I did specify transformer=DateFormatTransformer in entity and I did re-index :-(. Bit strange output. Even I tried to change the Data type of the data field in schema.xml as well. I have tried with changing date type from String to Date and Date to String by applying DateFormatTransformer. No luck :-(. -- View this message in context: http://lucene.472066.n3.nabble.com/Storing-MYSQL-DATETIME-field-in-solr-as-String-tp4106836p4106852.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Difference between textfield and strfield
Hey Iori, Apologize for misunderstanding :-). Yes agree with you, faceting will be OK with TextField type however I'm concern about performance impact while running the facets if we have millions of documents. I wish in future we could apply tokensizers and filters to String fields. :-).Thanks for your inputs. -- View this message in context: http://lucene.472066.n3.nabble.com/Difference-between-textfield-and-strfield-tp3986916p4105841.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Difference between textfield and strfield
I don't understand. Use the field type *Ahmet* recommended. Who is Ahmet? -- View this message in context: http://lucene.472066.n3.nabble.com/Difference-between-textfield-and-strfield-tp3986916p4105570.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Prioritize search returns by URL path?
Could it be achieved using multiple request handlers? Example: http://localhost:8983/solr/my/wiki http://localhost:8983/solr/my/blog http://localhost:8983/solr/my/forum As we could configure config for each request handler to specify the query. It would be great if Solr supports to query those three request handlers together and combine the result set? -- View this message in context: http://lucene.472066.n3.nabble.com/Prioritize-search-returns-by-URL-path-tp4105023p4105622.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Difference between textfield and strfield
Hi, If we can not analyse string field, is there any other way to apply case insensitiveness. Ex: I have a field called *make* its values are Toyota, Honda, Chery etc. In Solr make data type is string and values are stored as they appear (Toyota, Honda, Chery). However, when user try to search for toyota (instead Toyota), Solr wont fetch any result set as it does not match the exact word Toyota. I thought of applying LowerCaseFilterFactory at query level, but as its string field, I won't be able to do that. Is there any way, as I would like to search the user with case insensitive.? -- View this message in context: http://lucene.472066.n3.nabble.com/Difference-between-textfield-and-strfield-tp3986916p4105095.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Difference between textfield and strfield
Hi Iori, Thank you replying, really appreciate that. My concern not to use *TextField* and I want to make use of *string* field. Reason is that I have 7 fields that I want to apply case-insensitiveness and all these fields are *facetable* fields. It would not be feasible if I change data type from String to TextField just to apply case-insensitiveness. However Solr document suggests that if we were to make fields facetable, its better to use String. Is there any way of achieving this using String itself?. Thank you, Manju -- View this message in context: http://lucene.472066.n3.nabble.com/Difference-between-textfield-and-strfield-tp3986916p4105238.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Prioritize search returns by URL path?
Hi Jim Glynn, KAMACI is correct. How do you discriminate your documents?. Jim and Kamaci, I do have the same situation where I will be boosting document regular basis and expect documents with higher score appears on top and lower one at the bottom. Here is my requirement. My entity name is LISTING. I have few 100k listings and I distinguish each listing by *bumped, featured and normal* listings by listing_type. My Implementation plan is that - I want all the bumped (paid listings) on top of featured and normal listings - I want all featured listings on top of normal listings - Normal listings always stays bottom of the list Again in bumped listings, when user pays more money, those listings must appear on top of other bumped listings? My Question is - Can it be done using boosting those specific listings and sort by the score - Can it be done on the fly from an application and directly inject to Solr and reflect the changes in application again - At certain conditions, I might need to mix up bumped listings to get random bumped listings. -- View this message in context: http://lucene.472066.n3.nabble.com/Prioritize-search-returns-by-URL-path-tp4105023p4105245.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Multi-core support for indexing multiple servers
Hi Rob, mlti-core approach is different. You could have two cares lets say marketing-core [Has its own schema.xml and data-config.xml] magento-core [Has its own schema.xml and data-config.xml] each core have their own schema.xml and data-config.xml If you go by multi-core approach I guess you won't be able to achieve what you described or what you needed. You can query across two cores but that is expensive and tedious. The one you explained with having document type is just single core (Single index) and you differentiate each document by their type lets say document_type=marketing OR document_type=magento I think you could go by having single-index (Single-core) with document_type as differentiator. Also note that if you have common fields between two databases, you don't need to re-define those fields. You can make use of the same field for two databases. Lets say you have field 'title' in marketing database and magento database. You could have one 'title' field defined in schema.xml, no need to define two title fields. Also carefully look at each fields default values in schema.xml Lets say you have some fields in marketing database and those fields does not exists in magento db. When your done with indexing, if the fields does not have values they will not show up in the result. If you want it that way you don't need to define default=. If you still want to appear the field regardless of data or no data you would have to mention default= Ex: field name=year type=int indexed=true stored=true multiValued=false bdefault=*/ To index two databases together, you can try with DataImportHandler. In DataImportHandler you can query multiple data sources. Good thing about DataImportHandler is that your datasource could be data bases (MySQL, MS-SQL, etc), URLDataSource etc. Hope that is helpful -- View this message in context: http://lucene.472066.n3.nabble.com/Multi-core-support-for-indexing-multiple-servers-tp4099729p4099746.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Multi-core support for indexing multiple servers
Eric, Just a question :-), wouldn't it be easy to use DIH to pull data from multiple data sources. I do use DIH to do that comfortably. I have three data sources - MySQL - URLDataSource that returns XML from an .NET application - URLDataSource that connects to an API and return XML Here is part of data-config data source settings dataSource type=JdbcDataSource name=solr driver=com.mysql.jdbc.Driver url=jdbc:mysql://localhost/employeeDB batchSize=-1 user=root password=root/ dataSource name=CRMServer type=URLDataSource encoding=UTF-8 connectionTimeout=5000 readTimeout=1/ dataSource name=ImageServer type=URLDataSource encoding=UTF-8 connectionTimeout=5000 readTimeout=1/ Of course, in application I do the same. To construct my results, I do connect to MySQL and those two data sources. Basically we have two point of indexing - Using DIH at one time indexing - At application whenever there is transaction to the details that we are storing in Solr. -- View this message in context: http://lucene.472066.n3.nabble.com/Multi-core-support-for-indexing-multiple-servers-tp4099729p4099933.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to set default values for int fields
Erick, Thanks for replying :-). If I were to do that, we are trying to set string value to int and Solr throws an error. field name=mileage type=int indexed=true stored=true multiValued=false default=5/ Oh wait, I guess it works because Solr would automatically parse based on the data type of the field. :-). As I could see from the exception java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) *at java.lang.Integer.parseInt(Integer.java:493) * at java.lang.Integer.parseInt(Integer.java:514) at org.apache.solr.schema.TrieField.createField(TrieField.java:592) Thanks for the answer. I will test it as soon as I could reproduce the issue again -- View this message in context: http://lucene.472066.n3.nabble.com/How-to-set-default-values-to-int-fields-tp4099741p4099934.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: get min-max prices as facets
Hey Peter, Now I understood your requirement :-). I come across this http://wiki.apache.org/solr/StatsComponent have a look if it helps you :-). -- View this message in context: http://lucene.472066.n3.nabble.com/get-min-max-prices-as-facets-tp4099501p4099737.html Sent from the Solr - User mailing list archive at Nabble.com.
How to set default values for int fields
How do I set default value for int fields ex field name=mileage type=int indexed=true stored=true multiValued=false default=/ While indexing lets say if I have not set the value for mileage, and solr tries to assign default value and it fails. Because *mileage* is of type int not string. It throws following exception 3656198 [http-bio-8080-exec-4] ERROR org.apache.solr.servlet.SolrDispatchFilter – null:java.lang.NumberFormatException: For input string: at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:493) at java.lang.Integer.parseInt(Integer.java:514) at org.apache.solr.schema.TrieField.createField(TrieField.java:592) at org.apache.solr.schema.TrieField.createFields(TrieField.java:655) at org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:186) at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:324) at org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:73) at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:208) at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51) at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:519) at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:655) at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:398) at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100) at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:246) at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:173) at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1820) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:656) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:359) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:155) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:225) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1001) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:579) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:679) Also if I were to set default values for sting fields, it would be *default=null and not null*. is there way to set default value of the field to *null instead of null* is there any solution? -- View this message in context: http://lucene.472066.n3.nabble.com/How-to-set-default-values-for-int-fields-tp4099741.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr sort facets by name
Yes it is facet.sort=index would return facet result set in alphabetical order -- View this message in context: http://lucene.472066.n3.nabble.com/solr-sort-facets-by-name-tp4099499p4099522.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: get min-max prices as facets
I'm not sure if my answer would help you :-). Usually we do not need to know the min and max prices that our current database or solr is holding for all the products. Even if you were to that, would be complex and just make your business logic bigger and tedious. Instead, we would know what is the min price and what would be the max price. Based on this you could create price buckets in solr using http://wiki.apache.org/solr/SimpleFacetParameters#Facet_by_Range In others words, Lets say my site is about selling cars, I would mention min price could be =0 and max price = 400 (4million just logical value) could be more than that. So price range will fall between those two logically defined fields. Based on that you could construct solr facet query range, that Solr will automatically give you the price range (price bucket what amazon calls it). Hope that is helpful :-). -- View this message in context: http://lucene.472066.n3.nabble.com/get-min-max-prices-as-facets-tp4099501p4099523.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr starting problem after spellchecker configuration
Hi, Following post help you. http://stackoverflow.com/questions/7512945/solr-error-opening-new-searcher-exceeded-limit-of-maxwarmingsearchers-2-try Thanks -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-starting-problem-after-spellchecker-configuration-tp4087766p4087788.html Sent from the Solr - User mailing list archive at Nabble.com.
multiValued : How do I specify name to each value returned by multiValue field.
Hi All, I have a multiValued field, and it returns the result in the following manner doc str name=fnameMicheal/str str name=lnameSchumacher/str arr name=phones str090933434343/str str052323232323/str str073628362782/str /arr /doc In the above document phones is multiValued and it works fine :-). However I would like to name each row in the 'phones' with meaningful names like doc str name=fnameMicheal/str str name=lnameSchumacher/str arr name=phones *str name=mobile*090933434343/str *str home*052323232323/str *str office*073628362782/str /arr /doc any suggestions -- View this message in context: http://lucene.472066.n3.nabble.com/multiValued-How-do-I-specify-name-to-each-value-returned-by-multiValue-field-tp4083922.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: multiValued : How do I specify name to each value returned by multiValue field.
Hi Tamanjit, Not really :-). **phones** is an multiValued. Usually solr will produce list of array of values without naming each row something like this doc str name=fnameMicheal/str str name=lnameSchumacher/str arr name=phones str090933434343/str str052323232323/str str073628362782/str /arr /doc I wanted to be this way. doc str name=fnameMicheal/str str name=lnameSchumacher/str arr name=phones str name=mobile090933434343/str str name=home052323232323/str str name=office073628362782/str /arr /doc tamanjit.bin...@yahoo.co.in wrote So your doc would look something like this: doc str name=fname Micheal /str str name=lname Schumacher /str str name=mobile 090933434343 /str str name=home 052323232323 /str str name=office 073628362782 /str arr name=phones str 090933434343 /str str 052323232323 /str str 073628362782 /str /arr /doc tamanjit.bin...@yahoo.co.in wrote So your doc would look something like this: doc str name=fname Micheal /str str name=lname Schumacher /str str name=mobile 090933434343 /str str name=home 052323232323 /str str name=office 073628362782 /str arr name=phones str 090933434343 /str str 052323232323 /str str 073628362782 /str /arr /doc tamanjit.bin...@yahoo.co.in wrote So your doc would look something like this: doc str name=fname Micheal /str str name=lname Schumacher /str str name=mobile 090933434343 /str str name=home 052323232323 /str str name=office 073628362782 /str arr name=phones str 090933434343 /str str 052323232323 /str str 073628362782 /str /arr /doc -- View this message in context: http://lucene.472066.n3.nabble.com/multiValued-How-do-I-specify-name-to-each-value-returned-by-multiValue-field-tp4083922p4084149.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: multiValued : How do I specify name to each value returned by multiValue field.
Hi Jack, Thank you for the suggestions. I'm using DIH and the field names mobile,home,office comes as database fields. And 'phones' field is defined in schema.xml as multiValued. in DIH I'm using sub-entity for 'phones' fields. Let me give a try based on your suggestion :-). Thank you -- View this message in context: http://lucene.472066.n3.nabble.com/multiValued-How-do-I-specify-name-to-each-value-returned-by-multiValue-field-tp4083922p4084150.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr design. Choose Cores or Shards?
Hi Eric, Thanks for your reply. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-design-Choose-Cores-or-Shards-tp4082930p4083178.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr design. Choose Cores or Shards?
Hi, I have a confusion over choosing Cores or Shards for the project scenario. My scenario is as follows I have three entities 1. Customers 2. Product Info 3. Listings [Contains all the listings posted by customer based on product] I'm planning to design Solr structure for the above scenario like this 1. Customers Core 2. Product Info Core 3. Listings Core 4. Searchable Listing Core [Indexing searchable parameters selected from Listings, Product Info and Customer entities]. Having in mind that there wouldn't be much updates to Customers, Product Info. There will be regular updates to Listings that in turn I need to update Searchable listings that I could manage it. My Confusion is it feasible to choose many cores or use shards. I do not have much experience on how shards works and why they are used for. I would like to know the suggestions :-) for the design like this. What are the implications if I were to choose to use many cores and handle stuff at application level calling different cores. Thanks -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-design-Choose-Cores-or-Shards-tp4082930.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: entity classification solr
Can you provide sample structure of the document with entities, how does the document look like?. As far as I can assume, you do not need to apply any filters. If you are entities are searchable include them in the fulltext or keyword research. Is your entities are part of the document and are they multivalued? do you want to keep the word combination 'peanut butter' while indexing? Thanks -- View this message in context: http://lucene.472066.n3.nabble.com/entity-classification-solr-tp4082923p4082931.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Problems with distributed MoreLikeThis
I'm not sure about the root cause in your case. However one thing to remember while MLT is that, *MLT does not work with integer fields*. In your case if 'catchall' is copyField and if you are trying to copy any integer values verify it again :-). Thanks -- View this message in context: http://lucene.472066.n3.nabble.com/Problems-with-distributed-MoreLikeThis-tp4082873p4082932.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: MultiValue
Hi All,
I managed to *solve* the issue I had posted earlier with respect to
multiValued.
Here is the Query suppose to configured this way in *data-config.xml *
Description: in the below, first query has associated table images. Each
person would have many images. Here the JSON/XML would return all the images
associated with the person in block.
document name=manju
entity name=list dataSource=manjudb query=select
member_id,MemberName FROM member
entity name=imagelist dataSource=manjudb query=SELECT
imagepath FROM
images WHERE member_id='${list.member_id}'
/entity
/entity
/document
and *schema.xml* for the above query fields looks like
field name=member_id type=int indexed=true stored=true
required=true/
field name=MemberName type=string indexed=true stored=true/
field name=imagepath type=string indexed=true stored=true
bmultiValued=true* /
Output:
response: {
numFound: 3,
start: 0,
docs: [
{
MemberName: Vettel,
member_id: 1,
_version_: 1434904021528739800
},
{
MemberName: Schumacher,
member_id: 2,
imagepath: [ //As you could see here that Three rows from the
table *images* returned as one JSON object.
c:images\\etc\\test.jpg,
c:images\\etc\\test211.jpg,
c:images\\etc\\test2343434.jpg,
C:manju
],
_version_: 1434904021541322800
},
{
MemberName: J.Button,
member_id: 3,
_version_: 143490402154342
}
]
}
}
Thanks
--
View this message in context:
http://lucene.472066.n3.nabble.com/MultiValue-tp4034305p4062863.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: multiValued schema example (SOLVED)
Hi All,
I managed to *solve* the issue I had posted earlier with respect to
multiValued.
Here is the Query suppose to configured this way in *data-config.xml *
Description: in the below, first query has associated table images. Each
person would have many images. Here the JSON/XML would return all the images
associated with the person in block.
document name=manju
entity name=list dataSource=manjudb query=select
member_id,MemberName FROM member
entity name=imagelist dataSource=manjudb query=SELECT
imagepath FROM
images WHERE member_id='${list.member_id}'
/entity
/entity
/document
and *schema.xml* for the above query fields looks like
field name=member_id type=int indexed=true stored=true
required=true/
field name=MemberName type=string indexed=true stored=true/
field name=imagepath type=string indexed=true stored=true
bmultiValued=true* /
Output:
response: {
numFound: 3,
start: 0,
docs: [
{
MemberName: Vettel,
member_id: 1,
_version_: 1434904021528739800
},
{
MemberName: Schumacher,
member_id: 2,
imagepath: [ //As you could see here that Three rows from the
table *images* returned as one JSON object.
c:images\\etc\\test.jpg,
c:images\\etc\\test211.jpg,
c:images\\etc\\test2343434.jpg,
C:manju
],
_version_: 1434904021541322800
},
{
MemberName: J.Button,
member_id: 3,
_version_: 143490402154342
}
]
}
}
Thanks
--
View this message in context:
http://lucene.472066.n3.nabble.com/multiValued-schema-example-SOLVED-tp4062209p4062864.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: MultiValue
Hi All,
I wanted to achieve something similar
[ { last_name : jain, training_skill:[c, c++, php,java,.net] }]
[{person_id : 111, documents :
[doc_1,doc_2,doc_3,doc4,doc5,doc6]}]
Meaning one person could have more than one document.
how do we need to write the query, and configure data fields.
My current data-config.xml is
dataConfig
dataSource type=JdbcDataSource name=ds-carlist
driver=com.mysql.jdbc.Driver url=jdbc:mysql://127.0.0.1/solr_db
user=root password=root/
document name=person
entity name=list dataSource=ds-person query=SELECT a.id,b.docs
FROM
person as a,docs as b
WHERE a.id = b.person_id
/entity
/document
/dataConfig
and data field declearion in schema.xml is
field name=id type=int indexed=true stored=true
required=true/
field name=docs type=string indexed=true stored=true
multiValued=true/
Currently I don't see any errors and the no output.
I'm bit confused how to configure above scenario in Solr.
Thank you
--
View this message in context:
http://lucene.472066.n3.nabble.com/MultiValue-tp4034305p4062788.html
Sent from the Solr - User mailing list archive at Nabble.com.
multiValue schema example
Hi,
I have two table to be indexed.
Table-1: Person [id,name,addr]
Table-2: Docs [id,doc_name,person_id]
Relationship is: One person can have many documents.
I would like to get json as follows
[ { id : name, address,[doc_id1, doc_id2, doc_id3,etc] }]
How would I configure this in Solr.
How would I write configure the queries in data-config.xml
I tried this way. Main query and there is sub-query that returns many rows
to main query.
document name=example
entity name=person dataSource=ds-person query=SELECT
id,name,address,'documents' FROM Person
entity name=docs dataSource=ds-person query=SELECT id as
doc_id FROM docs WHERE person_id='${person.id}'
/entity
/entity
/document
This is how my schema.xml data filed configuration
field name=id type=int indexed=true stored=true required=true
multiValued=false /
field name=name type=string indexed=true stored=true
omitNorms=true/
field name=address type=string indexed=true stored=true/
field name=documents type=int indexed=true stored=true
omitNorms=true multiValued=true/
field name=doc_id type=int indexed=true stored=true
omitNorms=true/
I couldn't succeed.
What is the way to achieve the above scenario.
Thanks in advance.
--
View this message in context:
http://lucene.472066.n3.nabble.com/multiValue-schema-example-tp4062209.html
Sent from the Solr - User mailing list archive at Nabble.com.
