Efficient query to obtain DF

Dear Solr-User Group, Can you please suggest efficient query for retrieving term to document frequency(df) of that term at shard index level? I know we can get term to df mapping by applying termVectors component

Query to obtain count of term vocabulary

Dear Solr-User Group, Can you please suggest me API to query the *count* of total term vocabulary in a given shard index for specified field? For example, in the reference image click here , count of total terms in the "terms" column

Distributed IDF configuration query

Dear Solr User Group, I am trying to configure distributed idf(global df) to a collection consisting of 3 shards. Listed below are the configurations applied, however, the debug-explain results still show the "idf" computed at shard index level. For example, indexed total 7 docs, 3 among

Re: Solr on HDFS: Streaming API performance tuning

If you could provide the json parse exception stack trace, it might help to predict issue there. On Fri, Dec 16, 2016 at 5:52 PM, Chetas Joshi wrote: > Hi Joel, > > The only NON alpha-numeric characters I have in my data are '+' and '/'. I > don't have any backslashes.

Re: Solr MapReduce Indexer Tool is failing for empty core name.

ecific configurations. The same > tool released with solr 4.10 distro is working correctly, it seems to be > some issue/ changes from solr 5 onwards. I have tested it for both solr 5.5 > & solr 6.2.1 and the behaviour remains same for both. > > Thanks, > Manan Sheth > ______

Re: Solr - Amazon like search

There's a ecommerce features checklist with what solr can do listed here https://lucidworks.com/blog/2011/01/25/implementing-the-ecommerce-checklist-with-apache-solr-and-lucidworks/ That should be good start and then there are some more other references links listed below, I would try all of

Re: (Newbie Help!) Seeking guidance in regards to Solr's suggestor and others

This issue is on solarium-client php code, which is likely not traversing further to pick results from collation tag of solr response. at line 190 https://github.com/solariumphp/solarium/blob/master/library/Solarium/QueryType/Suggester/Result/Result.php#L190 verify if this is issue and do pull

Re: error diagnosis help.

Are you indexing xml files through nutch? This exception purely looks like processing of in-correct format xml file. On Mon, Dec 12, 2016 at 11:53 AM, KRIS MUSSHORN wrote: > ive scoured my nutch and solr config files and I cant find any cause. > suggestions? > Monday,

Re: Solr MapReduce Indexer Tool is failing for empty core name.

It looks like command line tool that you are using to initiate index process, is expecting some name to solr-core with respective command line param. use -help on the command line tool that you are using and check the solr-core-name parameter key, pass that also with some value. On Tue, Dec 13,

Re: Solr on HDFS: increase in query time with increase in data

I think the shard index size is huge and should be split. On Wed, Dec 14, 2016 at 10:58 AM, Chetas Joshi wrote: > Hi everyone, > > I am running Solr 5.5.0 on HDFS. It is a solrCloud of 50 nodes and I have > the following config. > maxShardsperNode: 1 > replicationFactor:

Re: Apply patch steps and update solr with new patch

jar files > for any SolrJ program I happen to be working with too... > > Best, > Erick > > On Thu, Dec 1, 2016 at 1:52 PM, Reth RM <reth.ik...@gmail.com> wrote: > > Hi, > > > > I followed the below steps to apply a patch, but have issues, any > p

Apply patch steps and update solr with new patch

Hi, I followed the below steps to apply a patch, but have issues, any pointers to mistake or blogs to apply patch and update solr with patch, will be helpful. 1. git clone https://github.com/apache/lucene-solr.git 2. ant clean, ant compile ant idea 3. open project in idea(intellij) 4. apply

Re: Wildcard searches with space in TextField/StrField

ery in debug mode does show 'field:a b*'. I am > sure > >> there are documents that should match. > >> Another (maybe unrelated) observation is if I have 'field:a\ b', then > the > >> parsedQuery is field:a field:b. Which does not match as expected > (matches >

Re: Editing schema and solrconfig files

whole thing back... I wrote a simple bash script to automate the scp > > process on both ends once I got tired of typing it over and over... > > > > On Wed, Nov 9, 2016 at 3:05 PM, Reth RM <reth.ik...@gmail.com> wrote: > > > >> What are some easiest ways to edit/mo

Re: Wildcard searches with space in TextField/StrField

I don't think you can do wildcard on StrField. For text field, if your query is "category:(test m*)" the parsed query will be "category:test OR category:m*" You can add q.op=AND to make an AND between those terms. For phrase type wild card query support, as per docs, it is

Re: SolrCloud Configuration

The easiest way is to create a /lib directory under each solr node and place the custom jar in it. But I think it doesn't get distributed over cluster, so this approach requires jar to be placed manually on each node. IIRC, it was recommended that such custom jar to be on disc than uploaded to

Editing schema and solrconfig files

What are some easiest ways to edit/modify/add conf files, such as solrconfig.xml and schema.xml other than APIs end points or using zk commands to re-upload modified file? In other words, can we edit conf files through solr admin (GUI) interface(add new filed by click on button or add new request

Re: For TTL, does expirationFieldName need to be indexed?

Yes, I think the field has to be indexed. If I understand correctly, DocExpirationUpdateProcessorFactory uses this field as query field, so it should be indexed=true. On Mon, Oct 17, 2016 at 11:35 AM, Brent wrote: > In my solrconfig.xml, I have: > > > >

Re: Sharding strategies

If you will have numerous documents, splitting documents into shard is a strategy. This split is independent of lingo of document. For documents with different languages, its necessary to use language specific analyzers to obtain good search results. For example, assume you have english language

Re: Distributing nodes with the collections API RESTORE command

Which version of solr? Afaik, until 6.1, solr backup and restore command apis required to do separate backup for each shard, and then restore in similar lines( both go for each). 6.1 version seems to have new feature of backing up entire collection records and then restoring it back to new

Re: Exception is thrown when using TimestampUpdateProcessorFactory

Hi Preeti, Try adding a default attribute to the solrtimestamp field in schema and check if this resolves the issue. replace with correct default date format https://cwiki.apache.org/confluence/display/solr/Defining+Fields On Thu, Sep 15, 2016 at 5:32 AM, preeti kumari

Re: Migrate data from solr4.9 to solr6.1

>>Is there any way through which I can migrate my index which is currently on 4.9 to 6.1? you should try copying existing indexes to latest solr 6.x and execute optimize command. Let us know your findings. >>I would be using solrcloud on solr 6.1.0 and will be having more number of shards than my

Re: language configuration in update extract request handler

This question should be posted on tika mailing list. It is not related to index or search but about parsing content of image. On Sun, Jun 5, 2016 at 10:20 PM, SIDDHAST® Roshan wrote: > Hi All, > > we are using the application for indexing and searching text using > solr. we

Re: Indexing a (File attached to a document)

Could you please let us know which crawler are you using to fetch data from document and its attachment? On Thu, May 12, 2016 at 3:26 PM, Solr User wrote: > Hi > > If I index a document with a file attachment attached to it in solr, can I > visualise data of that

Re: Multi-word Synonyms Solr 4.3.1 does not work

Right, this is a known issue. There is currently an active jira that you may like to watch https://issues.apache.org/jira/browse/SOLR-5379 And other possible workaround is explained here :

Re: Error - Too many close [count:-1]

Could you please some more background to this issue. Was it reported while indexing or querying? What is the version of solr? On Sat, Apr 30, 2016 at 12:04 AM, Vipul Gupta wrote: > Solr team - Any pointers on fixing this issue ? > > [10:29:08] ERROR 0-thread-7

Re: ANN: Solr puzzle: Magic Date

Yes, these can be practice/interview questions. But, considering the specific example above, it seems like question is pertaining to plot syntactically error(?); it is not expected that developer/solr-user know right syntax or commands. What could be interesting is, questions related to cloud

Re: Build Java Package for required schema and solrconfig files field and configuration.

Hi Nitin, If I understand correctly, you have configured suggest component in solr instance. Solr instance is an independent java program and it will be running on its own when you start and stop. You cannot package solr/suggest component in your java application/project. You can use SolrJ apis

Re: Solr Cloud Indexing Performance degrades suddenly

What are the recent changes made to database or DIH? Version upgrade? Addition of new fields? co-location of db? On Tue, Apr 26, 2016 at 2:47 PM, preeti kumari wrote: > I am using solr 5.2.1 . > > > -- Forwarded message -- > From: preeti kumari

Re: concat 2 fields

Check if you have added the 'concatFields' definition as well in solrconfig.xml... How are you indexing btw? On Tue, Apr 26, 2016 at 12:24 PM, vrajesh wrote: > Hi, > i have added it to /update request handler as per following in > solrconfig.xml: > > >

Re: regarding filter on spell checker

Could you please the requirement with an example? Its not clear as to what you mean by index property. On Tue, Apr 26, 2016 at 8:54 AM, Adrita G wrote: > Hi > >I want to whether we can apply any filters on spell checker.My > requirement is like that I need to filter

Re: The Streaming API (Solrj.io) : id must have DocValues?

Hi, So, is the concern related to same field value being stored twice: with stored=true and docValues=true? If that is the case, there is a jira relevant to this, fixed[1]. If you upgrade to 5.5/6.0 version, it is possible to read non-stored fields from docValues index., check out. [1]

Re: concat 2 fields

It should be added to /update request handler. All the others that you have listed here are search request handlers, you should add this one to /update RH. On Mon, Apr 25, 2016 at 12:12 PM, vrajesh wrote: > in my solr config there are many requestHandler so i am confused in

Re: concat 2 fields

Have you added this new processor chain to update handler that you are using(as shown below)? myChain https://wiki.apache.org/solr/UpdateRequestProcessor#Selecting_the_UpdateChain_for_Your_Request On Thu, Apr 21, 2016 at 2:59 PM, vrajesh wrote: > to concatenating two

Re: Solr Max Query length

(POST) over the other (GET)? > > -Frank > > > > > On 4/22/16, 11:13 AM, "Reth RM" <reth.ik...@gmail.com> wrote: > > >Are you using get instead of post? > > > >https://dzone.com/articles/solr-select-query-get-vs-post > > > > &g

Re: Solr Max Query length

Are you using get instead of post? https://dzone.com/articles/solr-select-query-get-vs-post On Fri, Apr 22, 2016 at 8:12 PM, Kelly, Frank wrote: > I used SolrJ and wrote a test to confirm that the max query length > supported by Solr (by default) was 8192 in Solr 5.3.1

Re: Wildcard query behavior.

If you search for f:validat*, then I believe you will get same number of results. Please check. f:validator* is searching for records that have prefix "validator" where as field with stemmer which stems "validator" to "validate" (if this stemming was applied at index time as well as query time)

Re: dataimport db-data-config.xml

What are the errors reported? Errors can be either seen on admin page logging tab or log file under solr_home. If you follow the steps mentioned on the blog precisely, it should almost work http://solr.pl/en/2010/10/11/data-import-handler-%E2%80%93-how-to-import-data-from-sql-databases-part-1/

Re: Question on Solr JDBC driver with SQL client like DB Visualizer

output of command : org/apache/solr/client/solrj/io/sql/ META-INF/services/java.sql.Driver org/apache/solr/client/solrj/io/sql/ConnectionImpl.class org/apache/solr/client/solrj/io/sql/DatabaseMetaDataImpl.class org/apache/solr/client/solrj/io/sql/DriverImpl.class

Question on Solr JDBC driver with SQL client like DB Visualizer

Note: I followed the steps mentioned in the pdf attached on this Jira https://issues.apache.org/jira/browse/SOLR-8521 Page 11, the screenshot specifies to select a "solr-solrj-6.0.0-SNAPSHOT.jar" which is equivalent into "solr-solrj-6.0.0.jar" shipped with released version, correct? When I try

Re: Cache problem

the OS cache, this might be the problem, > right? > (because it seems to cache every field ==> so all the data returned by the > query) > > kr, > Bast > > > On 12/04/2016 08:19, Reth RM wrote: > > As per solr admin dashboard's memory report, solr jvm is not

Re: Facet heatmaps: cluster coordinates based on average position of docs

Can you please be bit more specific on what type of query are you making and what other values are you expecting, with example? If you know of specific jira for the use case, then you can write comments there. On Mon, Apr 11, 2016 at 5:54 PM, Anton K. wrote: > Anyone? >

Re: Cache problem

As per solr admin dashboard's memory report, solr jvm is not using memory more than 20 gb, where as physical memory is almost full. I'd set xms=xmx=16 gb and let operating system use rest. And regarding caches: filter cache hit ratio looks good so it should not be concern. And afaik, document

Re: Solrj API for Managed Resources

I think its best to use available APIs. Here are the list of apis for managing synonyms and stop words https://cwiki.apache.org/confluence/display/solr/Managed+Resources And this blog post with details

Re: Specify relative path to current core conf folder when it's originally relative to solr home

I think there are some root paths defined in solr.sh file that will be in bin directory. You can pick root directory variable from there and use it. Example in solrconfig.xml, there is a value as : " ${solr.install.dir:../../../..}" I think solr.install.dir is the root path and its definition is

Re: search design question

Why not copy the field values of category, title, features, spec into a common text field and then search on that field. Otherwise use a edismax query parser and search with user search string on all the above fields may be by boosting title, category and specs field in order to get relevant

Re: SolrCloud backup/restore

Yes. It should be backing up each shard leader of collection. For each collection, for each shard, find the leader and request a backup command on that. Further, restore this on new collection, in its respective shard and then go on adding new replica which will duly pull it from the newly added

Re: How to Get info about clusterstate in solr 5.2.1 just like ping request handler with distrib=true

Have you already looked at cluster status api? https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api18 On Tue, Apr 5, 2016 at 10:09 AM, preeti kumari wrote: > Hi, > > I am using solr 5.2.1 . We need to configure F5 load balancer with >

Re: How to implement Autosuggestion

There is a payload attribute but I'm not sure if this can be used for such use case. Lets wait for others contributors to confirm. Similar question posted here: http://stackoverflow.com/questions/32434186/solr-suggestion-with-multiple-payloads . If its just a category that you need then the work

Re: most popular collate spellcheck

May be open a jira under improvement. https://issues.apache.org/jira/login.jsp? On Sat, Apr 2, 2016 at 11:30 PM, michael solomon <micheal...@gmail.com> wrote: > Thanks, and what we can do about that? > On Apr 2, 2016 5:28 PM, "Reth RM" <reth.ik...@gmail.com> wrot

Re: most popular collate spellcheck

Afaik, such feature doesn't exist currently, but looks like nice to have. On Thu, Mar 31, 2016 at 8:33 PM, michael solomon wrote: > Hi, > It's possible to return the most popular collate? > i.e: > spellcheck.q = prditive analytiycs > spellcheck.maxCollations = 5 >

Re: How to implement Autosuggestion

Solr AnalyzingInfix suggester component: https://lucidworks.com/blog/2015/03/04/solr-suggester/ On Mon, Mar 28, 2016 at 7:57 PM, Mugeesh Husain wrote: > Hi, > > I am looking for the best way to implement autosuggestion in ecommerce > using solr or elasticsearch. > > I guess

Re: Problem in Issuing a Command to Upload Configuration

I think it should be "zkcli.bat" (all in lower case) that is shipped with solr not zkCli.cmd(that is shipped with zookeeper) solr_home/server/scripts/cloud-scripts/zkcli.bat -zkhost 127.0.0.1:9983 \ -cmd upconfig -confname my_new_config -confdir server/solr/configsets/basic_configs/conf On

Re: scottchu] How to rebuild master-slave multi-core with schema.xml from old verison in Solr 5.5

Hi Scott, It is same as how we would do in earlier versions of solr. On the master instance, include the replication handler definition with master configs(as shown below). optimize optimize schema.xml,stopwords.txt,elevate.xml And on the slave instance, add the master url under slave

Re: score mixing

If you are looking for boosting the score of the document based on the value of rank field then you can as well use field boosting. rank^10. For the other case of adding scores and rank values, using "function query" should serve the requirement.

Re: Solr to Production

Is that website deployed on same machine where solr is running? If not, check whether the port is being blocked due to firewall protection. What is the response message that you are receiving? On Sun, Mar 27, 2016 at 3:16 PM, Adel Mohamed Khalifa < a.moha...@saudisoft.com> wrote: > Hello All,

Re: Issue With Manual Lock

Hi Salman, The index lock error is generally reported when 2 cores are trying to share an index directory between more than one core or Solr instance. Please check if there are more than one of those cores pointing to same data directory. You can see dir path on "overview" tab admin page. On

Re: Merge two Solr documents into One

As far as I know, there are no such ootb available utils but I can be wrong and lets wait for others thoughts as well. Other ways of dealing with this requirement is to write a custom update processor : https://cwiki.apache.org/confluence/display/solr/Update+Request+Processors On Wed, Mar 23,

Re: Indexing multiple pdf's and partial update of pdf

Are you using apache tika parser to parse pdf files? 1) Solr support parent-child block join using which you can index more than one file data within document object(if that is what you are looking for)