RE: Highlighting brings in irrelevant words
Forgot to include: All the three fields used for highlighting are configured as
follows:
Thank You,
:Sagar
-Original Message-
From: Bade, Vidya (Sagar) [mailto:vb...@webmd.net]
Sent: Friday, September 30, 2016 2:12 PM
To: solr-user@lucene.apache.org
Subject: Highlighting brings in irrelevant words
Hi,
I am using Solr 4.10.2 following is my request:
defType(Parser)=edismax
fl=id,title,description,link_title
qf=title description keywords
pf=title description
hl=true
hl.fl=title,description,link_title
hl.q=lupus
q=lupus
I have two records in the index. Both about lupus. When I query using the
above, the highlights are returned but not just the query term but also other
irrelevant terms as shown below. The only synonym I have for lupus is
"lupus,SLE,systemic lupus erythematosus" and I don't have the words "Symptoms"
or "Sleep" in the synonyms file.
Can anyone tell me what I am doing wrong or how to fix this issue?
"highlighting":{
"07":{
"title":["What Are the Symptoms of Lupus?"],
"description":["Medical guide to the symptoms of lupus."],
"link_title":["Lupus: The Symptoms and Signs"]},
"09":{
"title":["Lupus and Sleep"],
"description":["Join this team to get the tips you need to sleep better
while living with lupus."],
"link_title":["Lupus and Sleep"]}}
Thank You,
Sagar
Highlighting brings in irrelevant words
Hi,
I am using Solr 4.10.2 following is my request:
defType(Parser)=edismax
fl=id,title,description,link_title
qf=title description keywords
pf=title description
hl=true
hl.fl=title,description,link_title
hl.q=lupus
q=lupus
I have two records in the index. Both about lupus. When I query using the
above, the highlights are returned but not just the query term but also other
irrelevant terms as shown below. The only synonym I have for lupus is
"lupus,SLE,systemic lupus erythematosus" and I don't have the words "Symptoms"
or "Sleep" in the synonyms file.
Can anyone tell me what I am doing wrong or how to fix this issue?
"highlighting":{
"07":{
"title":["What Are the Symptoms of Lupus?"],
"description":["Medical guide to the symptoms of lupus."],
"link_title":["Lupus: The Symptoms and Signs"]},
"09":{
"title":["Lupus and Sleep"],
"description":["Join this team to get the tips you need to sleep better
while living with lupus."],
"link_title":["Lupus and Sleep"]}}
Thank You,
Sagar
Re: Re[2]: Traversal of documents through network
(1) So,dispalying the content(traversal of documents) depends on my pagination ? If i specify all 500 documents to be dispalyed and first 10 on the first page and remaining on the other, that implies that all documents traverse through network ? (2) In my application, front end of UI is developed by one team and back end of indexing is done by one team. How to implement highlighting feature in solr. By developing a request handler with highlighting parameters turned on and with termvector components, can i highlight the term queried ? Or any special configuration needed ? Please help me on these two. Thanks -- View this message in context: http://lucene.472066.n3.nabble.com/Traversal-of-documents-through-network-tp4271555p4272053.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Traversal of documents through network
ok. I understand that. So, you would say documents traverse through network. If i specify some 100 docs to be dispalyed on my first page, will it effect performance. While docs gets traversed, will there be any high volume traffic and effects performance of the application. And whats the time solr takes to index 1GB of data in general. Thanks -- View this message in context: http://lucene.472066.n3.nabble.com/Traversal-of-documents-through-network-tp4271555p4271743.html Sent from the Solr - User mailing list archive at Nabble.com.
Traversal of documents through network
Hi When i queried a word in solr, documents having that keyword is displayed in 500 documents,lets say. Will all those documents traverse through network ? Or how it happens ? Please help me on this. -- View this message in context: http://lucene.472066.n3.nabble.com/Traversal-of-documents-through-network-tp4271555.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: indexing pdf files using post tool
Sorry for conveying it in wrong way. I want my data of 1 pdf file to be indexed with different fields in a document of solr according to data in it like name;id;title;content etc Thanks -- View this message in context: http://lucene.472066.n3.nabble.com/indexing-pdf-files-using-post-tool-tp4263811p4264052.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: indexing pdf files using post tool
Hi I got data into my content field. But i wanted to have differnt fields to be allocated for data in my file.How can I achieve this ? -- View this message in context: http://lucene.472066.n3.nabble.com/indexing-pdf-files-using-post-tool-tp4263811p4263840.html Sent from the Solr - User mailing list archive at Nabble.com.
indexing pdf files using post tool
Hi
I am trying to index a pdf file by using post tool in my linux system,When i
give the command
bin/post -c core2 -p 8984 /root/solr/My_CV.pdf
it is showing the search results like
"response": {
"numFound": 1,
"start": 0,
"docs": [
{
"id": "/root/solr-5.5.0/My_CV.pdf",
"meta_creation_date": [
"2016-03-15T06:22:17Z"
],
"pdf_pdfversion": [
1.4
],
"dcterms_created": [
"2016-03-15T06:22:17Z"
],
"x_parsed_by": [
"org.apache.tika.parser.DefaultParser",
"org.apache.tika.parser.pdf.PDFParser"
],
"xmptpg_npages": [
1
],
"creation_date": [
"2016-03-15T06:22:17Z"
],
"pdf_encrypted": [
false
],
"title": [
"My CV"
],
"stream_content_type": [
"application/pdf"
],
"created": [
"Tue Mar 15 06:22:17 UTC 2016"
],
"stream_size": [
18289
],
"dc_format": [
"application/pdf; version=1.4"
],
"producer": [
"wkhtmltopdf"
],
"content_type": [
"application/pdf"
],
"xmp_creatortool": [
"þÿ"
],
"resourcename": [
"/root/solr/My_CV.pdf"
],
"dc_title": [
"My CV"
],
"_version_": 1528851429701189600
}
but not the actual content in pdf file.
How to index that dat.
Please help me on this.
Can post tool be used for indexing data from HDFS ?
--
View this message in context:
http://lucene.472066.n3.nabble.com/indexing-pdf-files-using-post-tool-tp4263811.html
Sent from the Solr - User mailing list archive at Nabble.com.
accessing data in hdfs by solr in standalone mode
Hi can solr access the data from HDFS in standalone mode? If so, can u brief how it is done. Thnaks in advance -- View this message in context: http://lucene.472066.n3.nabble.com/accessing-data-in-hdfs-by-solr-in-standalone-mode-tp4263805.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Data Import Handler Usage
Hi Dataimport section in web ui page still shows me that no data import handler is defined. And no data is being added to my new collection. -- View this message in context: http://lucene.472066.n3.nabble.com/Data-Import-Handler-Usage-tp4257518p4257576.html Sent from the Solr - User mailing list archive at Nabble.com.
Data Import Handler Usage
Hi I have gone through documents to define data import handler in solr. But i couldnot implement it. I have created data-config.xml file that specifies moving data from collection1 core to another collection, i donno where i need to specify that second collection. http://localhost:8983/solr/collection1"; query="*:*"/> and request handler is defined as follows in solrconfig.xml /home/username/data-config.xml Even after adding this, i couldnot get any data import handler in web url page for importing. Why is it so? And what changes need to be done? I have followed the following url : http://www.codewrecks.com/blog/index.php/2013/4/29/loading-data-from-sql-server-to-solr-with-a-data-import-handler -- View this message in context: http://lucene.472066.n3.nabble.com/Data-Import-Handler-Usage-tp4257518.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr-kerbarose URL not accessible
Hi When I am trying to access my solrCloud web UI page, deployed in cloudera cluster, I have encountered with the error "DEFECTED TOKENS DETECTED" . Find the attachment of the error that is added here. It is because of kerbarose installed on cluster. Is there any other way that I can access solr in this scenario with kerbarose installed ? Writing a java program helps in any way? While writing a java program also, i have to give connection to solr URL with port or zookeeper host variable. Will that java program work out? Please help me out. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-kerbarose-URL-not-accessible-tp4256926.html Sent from the Solr - User mailing list archive at Nabble.com.
Solrj-collection creation
Hi I want to connect to solrCloud server from java program using zookeeperHost variable. I know that data can be indexed and searched from a collection using java program. but Can i able to create a collection initially from java program? My problem is that i cannot access solr web page, i'm getting an error like "defected tokens detected". So, i wanted to connect to solr using solrj API. For that I need to create a collection initially.Got struck here. Anyone can help me. Thanks in advance -- View this message in context: http://lucene.472066.n3.nabble.com/Solrj-collection-creation-tp4256380.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: Multi-lingual search
Hi Can i implement proximity search if i use >seperate core per language >field per language >multilingual field that supports all languages. And what does proximity search exactly mean? searching for walk word when walking is indexed,should fetch and display the record? It will be included in stemming filter.right? Thanks in advance -- View this message in context: http://lucene.472066.n3.nabble.com/Multi-lingual-search-tp4254398p4256094.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: Multi-lingual search
Hi I need to search on these languages which includes proximity search also. 1.Malay 2.Tamil 3.Bahasa Indonesia 4.Vietnamese 5.Cantonese Will IndicNormalizationFilter work fine or any other filter? Help me if you have already worked on it or have any idea. Thanks in advance -- View this message in context: http://lucene.472066.n3.nabble.com/Multi-lingual-search-tp4254398p4255826.html Sent from the Solr - User mailing list archive at Nabble.com.
Import data from one core to another
Hi How to import data from one solr core to another using request handler and data-config.xml? In solr-config.xml : I included this in target collection, /root/Desktop/vidya/solr-data-config.xml And solr-data-config.xml in the path as mentioned in request handler class http://localhost:8983/solr/";> Then i reloaded the collection and executed this query http://10.138.90.227:8983/solr/#/student_shard1_replica1/dataimport?command=full-import But i got an error as "NO DATA IMPORT HANDLER IS DEFINED" What is the query to fully import data from source to target? Thanks in advance -- View this message in context: http://lucene.472066.n3.nabble.com/Import-data-from-one-core-to-another-tp4254590.html Sent from the Solr - User mailing list archive at Nabble.com.
Multi-lingual search
Hi My use case is to index and able to query different languages in solr which are not in-built languages supported by solr. How can i implement this ? My input document consists of different languages in a field. I came across "Solr in action" book with searching content in multiple languages i.e., chapter 14. For built in languages i have implemented this approach. But for languages like Tamil, how to implement? Do i need to find for filter classes of that particular language or any libraries in specific. Please help me on this. Thanks in advance. -- View this message in context: http://lucene.472066.n3.nabble.com/Multi-lingual-search-tp4254398.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Restoring backups of solrcores
Hi How can that be useful, can u please explain. I want to have the same collection name everytime when I index data i.e., current_collection. By collection aliasing, i can create a new collection and point my alias (say ALIAS) to new collection but cannot rename that collection to the same current_collection which i have created and indexed previous week. So, are you asking me to create whatever collection name i want to create but point out my alias with name i want and change that alias pointing to new collection that i create and query using my alias name. Please help me on this. Thanks in advance -- View this message in context: http://lucene.472066.n3.nabble.com/Restoring-backups-of-solrcores-tp4254080p4254366.html Sent from the Solr - User mailing list archive at Nabble.com.
Restoring backups of solrcores
Hi I wanted to restore the core into a new one in solrcloud by following the below link. https://cwiki.apache.org/confluence/display/solr/Making+and+Restoring+Backups+of+SolrCores But I couldnt get the success value when restore is given. It cant be implemented for solrcloud? If so, then what is the alternative that can be suggested. My requirement is say, I have one collection called current_collection which has current weeks data. In the later week, I want to move the data in current_collection to previous_collection1 and index new week's data into current_collection by removing previous data and storing it in previous_collection1. How can I achieve this? Thanks in advance -- View this message in context: http://lucene.472066.n3.nabble.com/Restoring-backups-of-solrcores-tp4254080.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: collection aliasing
Hi How can a full index rebuild can be done ? -- View this message in context: http://lucene.472066.n3.nabble.com/collection-aliasing-tp4252527p4254077.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: collection aliasing
Hi Then what is the difference between collection aliasing and shards parameter mentioned in request handler of solrconfig.xml. In request handler of new collection's solrconfig.xml shards = http://localhost:8983/solr/collection1,http://localhost:8983/solr/collection1 I can query both data of collection1 and collection2 in new collection which is same as collection aliasing. Is my understanding correct ? If so, then what is the special characteristic of collection alaising. Please help me. Thanks in advance -- View this message in context: http://lucene.472066.n3.nabble.com/collection-aliasing-tp4252527p4253787.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: collection aliasing
Yeah, while querying and indexing also, we can directly use our collection names. Then what is the use of aliasing ? -- View this message in context: http://lucene.472066.n3.nabble.com/collection-aliasing-tp4252527p4252885.html Sent from the Solr - User mailing list archive at Nabble.com.
collection aliasing
Hi I wanted to mainatain two sets of indexes or collections for maintaing my large input data for indexing for which i found collection aliasing is helpful. I have created alais for 2 collections. but my problem is , how can i point out my alias to 2 different colletions at 2 different times. Thanks in advance. -- View this message in context: http://lucene.472066.n3.nabble.com/collection-aliasing-tp4252527.html Sent from the Solr - User mailing list archive at Nabble.com.
collection aliasing,solrctl
Hi I am using solr with cloudera distribution to index data from hdfs and I am using "solrctl" utility for my deployment. Now i wanted to create collection alias. How can i perform the action of creating collection aliasing by commands. >From google i got : " /admin/collections?action=CREATE " . How to achieve this by solrctl or by anything by commands from terminal. Thanks in advance -- View this message in context: http://lucene.472066.n3.nabble.com/collection-aliasing-solrctl-tp4252210.html Sent from the Solr - User mailing list archive at Nabble.com.
Error while reloading collection
Hi I am using solrcloud on cloudera cluster. I have created collections using solrctl command which is supported by cloudera search tool. I included one class of java in schema.xml for creating a field type which is dependent on a jar that i have included in solrconfig.xml. But when i reload that collection, I am getting an error that ERROR LOADING THAT CLASS what i included in schema.xml. What else do i need to include ? In solrconfig.xml : Error while relaoding in command line interface : 0 197 org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error handling 'reload' action Please help me on this.Thanks in advance -- View this message in context: http://lucene.472066.n3.nabble.com/Error-while-reloading-collection-tp4250635.html Sent from the Solr - User mailing list archive at Nabble.com.
collection reflection in resource manager node
Hi I have created a collection in one datanode on which solr server is deployed say DN1. I am having another datanode on which solr server is deployed which has resource manager service also running on it,say DN2. When i created a collection using solrctl command in DN1, it got reflected in DN2 but not DN1. Why is it so ? Please help me on this. If i need to put some jars for indexing in my collection,where do i need to put ? in DN! or DN2 Thanks in advance -- View this message in context: http://lucene.472066.n3.nabble.com/collection-reflection-in-resource-manager-node-tp4250382.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: solrcloud -How to delete a doc at a specific shard
So, you have deployed solr server on three nodes namely 192.168.100.210;211;212 . Am I correct ? -- View this message in context: http://lucene.472066.n3.nabble.com/solrcloud-How-to-delete-a-doc-at-a-specific-shard-tp4249354p4250117.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: solrcloud -How to delete a doc at a specific shard
Hi I am new to solr and having a doubt on how one can know that a particular shard is in that particular node or ip address. Thanks in advance -- View this message in context: http://lucene.472066.n3.nabble.com/solrcloud-How-to-delete-a-doc-at-a-specific-shard-tp4249354p4250072.html Sent from the Solr - User mailing list archive at Nabble.com.
solr in action - multiple language content in one field
Hi I have gone through solr in action 14th chapter which tells - "searching content in multiple languages" . But i have a doubt that when i put documents in solr web UI, it recognises every language and gives me the result when queried for it. What exactly did they depict in that chapter. can't solr recognise and process all languages at a time? -- View this message in context: http://lucene.472066.n3.nabble.com/solr-in-action-multiple-language-content-in-one-field-tp4250071.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: core,Collection,Shard,Replication
Hi You described that sharding is to distribute data over multiple machines.Do I have to make sure that all those machines have solr server or gateway should be deplyed ? And what multiple JVM processes run behind a solr server running? I wanted to know what is a node. -> I understood like a mchine with solr server deployed. then what is a solr instance? Am I correct.If not,please help me Thanks in advance -- View this message in context: http://lucene.472066.n3.nabble.com/core-Collection-Shard-Replication-tp4248850p4248865.html Sent from the Solr - User mailing list archive at Nabble.com.
core,Collection,Shard,Replication
Hi I am new to solr. i have a doubt in understanding difference between core and collection. >As far as i understand, cores can be created when solr is run in local mode and collections in solrCloud. Can you please help me if i am wrong. >And why do we shard a collection? i read like - When your data is too large for one node, you can break it up and store it in sections by creating one or more shards. Each is a portion of the logical index, or core, and it's the set of all nodes containing that section of the index. But when indexing a document in one shard,it gets reflected in every shard of that collection. But main intention of creating shard is to break up the data. >Why do we replicate a collection? Thanks in advance -- View this message in context: http://lucene.472066.n3.nabble.com/core-Collection-Shard-Replication-tp4248850.html Sent from the Solr - User mailing list archive at Nabble.com.
Data migration from one collection to the other collection
Hi I would like to maintain two cores for history data and current data where hdfs is my datasource. My requirement is that data input should be given to only one collection and previous data should be moved to history collection. 1)Creating two cores and migrating data from current to history collection by data-config.xml using solrEntityProcessor. In data-config.xml, where should i represent two collections for migrating source collection to the other collection.And how to make sure that happens.Do I need to run a job or how to make sure that data migration occurs. https://wiki.apache.org/solr/DataImportHandler#SolrEntityProcessor 2)collection aliasing is a concept which creates new collection after a period of time. I read that concept but lagging in how to implement it.Like where do i need to make changes in my solrcloud. http://blog.cloudera.com/blog/2013/10/collection-aliasing-near-real-time-search-for-really-big-data/ Please help me on this. Thanks in advance -- View this message in context: http://lucene.472066.n3.nabble.com/Data-migration-from-one-collection-to-the-other-collection-tp4248646.html Sent from the Solr - User mailing list archive at Nabble.com.
Querying with action parameter included in URL
Hi I am pretty new to solr and when i am going through the tutorials , I came across urls for querying like "http://localhost:8983/solr/admin/configs?action=CREATE&name=booksConfig&baseConfigSet=genericTemplate"; . I wanted to know how to implement the same by doing changes in schema.xml or solrconfig.xml. Where should i make changes when an "action=" is specified. Thanks in advance -- View this message in context: http://lucene.472066.n3.nabble.com/Querying-with-action-parameter-included-in-URL-tp4248576.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: MapReduceIndexerTool Indexing
Hi I would like to index only new data but not already indexed data(delta Indexing). how can i achieve it using MRIT. Thanks in advance -- View this message in context: http://lucene.472066.n3.nabble.com/MapReduceIndexerTool-Indexing-tp4248387p4248573.html Sent from the Solr - User mailing list archive at Nabble.com.
MapReduceIndexerTool Indexing
Hi I have used MapReduceIndexerTool to index data in my hdfs to solr inorder to search it. I want to know whether it indexes entire data when some new data is added to that path, again when tool is run on it. Thanks in advance -- View this message in context: http://lucene.472066.n3.nabble.com/MapReduceIndexerTool-Indexing-tp4248387.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: Using the date field for searching
You can use filter query and form the date as follows when a user enters just the year or year and month: If just the year (1885) was entered -> date:[1885-01-01T00:00:00Z TO 1886-01-01T00:00:00Z] If just the year and month (1885-06) were entered -> date:[1885-06-01T00:00:00Z TO 1885-07-01T00:00:00Z] Alternatively use DateRangeField as described at the bottom in the following webpage: https://cwiki.apache.org/confluence/display/solr/Working+with+Dates :Sagar -Original Message- From: Scott Derrick [mailto:sc...@tnstaafl.net] Sent: Tuesday, August 11, 2015 3:02 PM To: solr-user@lucene.apache.org Subject: Using the date field for searching If I query date:1885 I get an error org.apache.solr.common.SolrException: Invalid Date String:'1885' If I query date:1885* I get no results. and yet there are numerous docs with a year of 1885 in the date string, like so 1885-02-08T00:00:00Z if I query date:1885-02-08T00:00:00Z I get 9 results?? Do the users really have to specify a full xml compliant date string to use the date: field for searching? thanks, Scott
Help with separate root entities in DIH - One each for full and delta import.
Hi, I am currently using Solr 4.10.2 and having issues with Delta-imports. For some reason delta seems to be inconsistent when using query caching. I am using SqlEntityProcessor. To overcome the issue I want to try having two root entities - one each for full and delta. Can someone help with a sample format. Following is my current data import setup: Thank You, Sagar
