setRAMBufferSizeMB and setRAMPerThreadHardLimitMB

each thread reaches 1945 MB? -- Gimantha Bandara Software Engineer WSO2. Inc : http://wso2.com Mobile : +94714961919

Re: Lucene 5.0.0 - StringField and Sorting

en > > > > > > > > > > > > - > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > > > -- Gimantha Bandara Software Engineer WSO2. Inc : http://wso2.com Mobile : +94714961919

Re: Lucene 5.0.0 - StringField and Sorting

--- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > > -- Gimantha Bandara Software Engineer WSO2. Inc : http://wso2.com Mobile : +94714961919

Re: GROUP BY in Lucene

gt; wrote: > > > Lucene has a grouping module that has several approaches for grouping > > search hits, though it's only by a single field I believe. > > > > Mike McCandless > > > > http://blog.mikemccandless.com > > > > > > On Sun, A

Re: GROUP BY in Lucene

ader().getBinaryDocValues(this.indexFieldName); > > //Here find the oridinals of the group-by-fields and the > arrgegate fields. > //Create a multi ordinal key MultiIntKey from the > group-by-ordinals and use that to add the current value of the fiels to do > the agggregation to the facet-aggregates > > .. > > > Hope this helps :) > -Rob > > -- Gimantha Bandara Software Engineer WSO2. Inc : http://wso2.com Mobile : +94714961919

How to get the top facets values of a field/dimension which has the highest number of immediate children

Hi, Lets say I have thousands of Lucene documents which have a FacetField which has the format of the following. doc.add(new FacetField("Category", "level0", "level1", "level2", "level3", ...)); "Category" is the dimension name. "level0" represents the first hierarchical level, "level1"

Re: How does sorting work in Lucene?

be compared to another document's score of a different index). If we assume that the score is properly returned, can I use the scores of the docs of different indices and sort all the doc ids from all the indexes( probably using mergesort)? On Tue, Mar 1, 2016 at 11:04 AM, Gimantha Bandara <gi

Getting an Exception while searching when (numHits = Large Number) in TopScoreDocCollector

I know that I am getting this exception because the priorityQueue allocate memory more than my PC can allocate from the RAM. ERROR {org.wso2.carbon.analytics.dataservice.core.indexing.AnalyticsDataIndexer} - Error in index search: null java.lang.NegativeArraySizeException at

Re: How does sorting work in Lucene?

Any thoughts? On Monday, February 29, 2016, Gimantha Bandara <giman...@wso2.com> wrote: > Hi all, > > We are using lucene to index our data and are maintaining millions of > documents in sharded indices. Currently what we do is, reading each shard > separately and collect

How does sorting work in Lucene?

Lucene support pagination? Help is much appreciated. -- Gimantha Bandara Software Engineer WSO2. Inc : http://wso2.com Mobile : +94714961919

Re: Equivalent API in Lucene 5.x.x for ResultMode.setResultMode and ResultMode.setDepth

any help on this please? On 10/6/15, Gimantha Bandara <giman...@wso2.com> wrote: > Hi, > > I want to get the top categories (all the categories) recursively in one > call. I know that in Lucene 4.x.x we can simply set the ResultMode to > PER_NODE_IN_TREE [1] and set the

Equivalent API in Lucene 5.x.x for ResultMode.setResultMode and ResultMode.setDepth

Hi, I want to get the top categories (all the categories) recursively in one call. I know that in Lucene 4.x.x we can simply set the ResultMode to PER_NODE_IN_TREE [1] and set the depth, so we can get the categories recursively to the level we want in the tree. How do I achieve the same in Lucene

Re: Document updates work as delete/add under the hood

as it feels like a real bottleneck. Is there any way to do single fields updates without using delete/add ? Thanks, Chalitha -- J.M Chalitha Udara Perera *Department of Computer Science and Engineering,* *University of Moratuwa,* *Sri Lanka* -- Gimantha Bandara Software Engineer WSO2

Re: Document updates work as delete/add under the hood

need to update only a single field. Re-indexing a new document with updated field + other fields seems to be big overhead. My question is, why lucene does that and currently is there a way we can avoid this ? Thanks, Chalitha On Fri, Jul 10, 2015 at 1:46 PM, Gimantha Bandara giman...@wso2

Re: Using lucene queries to search StringFields

at 10:45 AM, Ahmet Arslan iori...@yahoo.com.invalid wrote: Hi, Why don't you create your query with API? Term term = new Term(B, 1 2); Query query = new TermQuery(term); Ahmet On Friday, June 19, 2015 9:31 AM, Gimantha Bandara giman...@wso2.com wrote: Correction

Re: Using lucene queries to search StringFields

the correct way to query for StringFields? Thanks, On Thu, Jun 18, 2015 at 2:12 PM, Gimantha Bandara giman...@wso2.com wrote: Hi all, I have created lucene documents like below. Document doc = new Document(); doc.add(new TextField(A, 1, Field.Store.YES)); doc.add(new StringField(B, 1 2 3

Using lucene queries to search StringFields

using whitespace analyzer everywhere) -- Gimantha Bandara Software Engineer WSO2. Inc : http://wso2.com Mobile : +94714961919

Re: Exception While searching through indices.

/wso2/carbon/analytics/dataservice/indexing/AnalyticsDataIndexer.java On Sun, Jun 14, 2015 at 7:13 PM, Đạt Cao Mạnh caomanhdat...@gmail.com wrote: Can you post you scenario in detail along with your modification please? On 14:09, Sun, 14 Jun 2015 Gimantha Bandara giman...@wso2.com wrote: Hi Dat

Re: Exception While searching through indices.

is Integer.MAX_VALUE. So using a single lucene index to index billions documents is not a proper ways. You should consider using Solr Cloud or Elasticsearch to index your documents. On 19:43, Fri, 12 Jun 2015 Gimantha Bandara giman...@wso2.com wrote: Hi all, We are using Lucene 4.10.3 for indexing

Exception While searching through indices.

) -- Gimantha Bandara Software Engineer WSO2. Inc : http://wso2.com Mobile : +94714961919

Re: Migrating from Lucene 4.10.3 to Lucene 5.10

return a Lock instance. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Gimantha Bandara [mailto:giman...@wso2.com] Sent: Thursday, May 21, 2015 3:12 PM To: java-user@lucene.apache.org

Migrating from Lucene 4.10.3 to Lucene 5.10

it in 5.1.0. How does Lucene 5.1.0 handle releasing the lock? -- Gimantha Bandara Software Engineer WSO2. Inc : http://wso2.com Mobile : +94714961919

Re: Exception while updating a lucene document

...@gmail.com wrote: seems like you forgot to do facetsConfig.setMultiValued(`field`, true) too . On Sat, Apr 25, 2015 at 7:37 AM, Gimantha Bandara giman...@wso2.com wrote: Hi, I was able to fix the problem.. the issue was with my wrong usage of FacetConfig class. I was creating Document using

Re: Exception while updating a lucene document

to the document, Call the facetConfig.build at last. Then call the updateDocument on the created document. On Fri, Apr 24, 2015 at 10:13 PM, Gimantha Bandara giman...@wso2.com wrote: Hi all, I have documents which have some facetfields. If I tried to update a document where the facet values are same

Exception while updating a lucene document

Hi all, I have documents which have some facetfields. If I tried to update a document where the facet values are same in both facet fields i am getting the following error. Note I am using updateDocument in indexWriter to create the document. Exception in thread pool-23-thread-2

Joining two Indices in Lucene

Hi, I am now looking into BlockJoinQuery where I can join two indices and execute search queries. My concern is this... Is it possible to perform AND and OR operations between two seperate indices? As an example, I have a common field _id in both indices. I will relate the two indices( using

How to read multiple indices in parallel.

Hi all, As I can see the Multireader is reading the multiple indices sequentially (correct me if I am wrong). So using a IndexSearcher on a multireader will also perform sequential searches right? Is there a lucene-built-in class to search several indices parallely? -- Gimantha Bandara Software

Re: How to read multiple indices in parallel.

Collector). The not-yet-released Lucene 5.1 includes some changes (LUCENE-6294 https://issues.apache.org/jira/browse/LUCENE-6294) that enable better parallel query support. --Terry On Tue, Apr 7, 2015 at 8:09 AM, Gimantha Bandara giman...@wso2.com wrote: Hi all, As I can see

Re: How to read multiple indices in parallel.

on (A, B) would use leaves B, C, E, F. This runs serially by default but can run in parallel if you provide an ExecutorService to the IndexSearcher and use a compatible search() method on it. --Terry On Tue, Apr 7, 2015 at 10:27 AM, Gimantha Bandara giman...@wso2.com wrote: Hi Terry, I

Re: for check similarity of two sentences

those questions.can i do that using Apache Lucene? Thank You! regards, Heshan jayasinghe -- Gimantha Bandara Software Engineer WSO2. Inc : http://wso2.com Mobile : +94714961919

Re: How to merge several Taxonomy indexes

Btw I was using a RAMDirectory for just testing purposes.. On Thu, Apr 2, 2015 at 5:16 PM, Gimantha Bandara giman...@wso2.com wrote: Hi Christoph and Shai, Thanks for the quick response!. Indices are stored in a relational database ( using a custom Directory implementation ). The Problem

Re: How to merge several Taxonomy indexes

handle this issue? On Tue, Mar 24, 2015 at 12:18 PM, Gimantha Bandara giman...@wso2.com wrote: Hi Christoph, My mistake. :) It does the exactly what i need. figured it out later.. Thanks a lot! On Tue, Mar 24, 2015 at 3:14 AM, Gimantha Bandara giman...@wso2.com wrote: Hi Christoph, I

Re: How to merge several Taxonomy indexes

a RAMDirectory? If your merged index fits into RAM completely, a MMapDirectory should offer almost the same performance. And if not, it is definitely the better choice. Regards Christoph Am 02.04.2015 um 12:38 schrieb Gimantha Bandara: Hi All, I have successfully setup a merged

Re: How to merge several Taxonomy indexes

Bandara giman...@wso2.com wrote: Btw I was using a RAMDirectory for just testing purposes.. On Thu, Apr 2, 2015 at 5:16 PM, Gimantha Bandara giman...@wso2.com wrote: Hi Christoph and Shai, Thanks for the quick response!. Indices are stored in a relational database ( using

Re: Would Like to contribute to Lucene

. I would recommend having a look at Mike's blog http://blog.mikemccandless.com/ and videos of Lucene Revolution, ApacheCon and BerlinBuzzwords which regularly get a fair amount of Lucene/Solr-related talks. On Fri, Mar 13, 2015 at 6:54 AM, Gimantha Bandara giman...@wso2.com wrote

Getting the doc values grouped by Facets

grouped by facets. Is it possible? -- Gimantha Bandara Software Engineer WSO2. Inc : http://wso2.com Mobile : +94714961919

Re: How to merge several Taxonomy indexes

Hi Christoph, My mistake. :) It does the exactly what i need. figured it out later.. Thanks a lot! On Tue, Mar 24, 2015 at 3:14 AM, Gimantha Bandara giman...@wso2.com wrote: Hi Christoph, I think TaxonomyMergeUtils is to merge a taxonomy directory and an index together (Correct me if I am

How to merge several Taxonomy indexes

. It is easy. To Merge document indexes, all I have to do is create a MultiReader and pass it to IndexSearcher. But I am stuck at merging the taxonomy indexes. Is there a way to merge taxonomy indexes? -- Gimantha Bandara Software Engineer WSO2. Inc : http://wso2.com Mobile : +94714961919

Re: How to merge several Taxonomy indexes

: Hi Gimantha, have a look at the class org.apache.lucene.facet.taxonomy.TaxonomyMergeUtils, which does exactly what you need. Best regards, Christoph Am 23.03.2015 um 15:44 schrieb Gimantha Bandara: Hi all, Can anyone point me how to merge several taxonomy indexes? My requirement

Re: Would Like to contribute to Lucene

Any clue on where to start from? On Fri, Mar 13, 2015 at 11:24 AM, Gimantha Bandara giman...@wso2.com wrote: Hi all, I am willing to contribute to Lucene project. I have already been referring to Lucene in Action 2nd edition recently. But I think it is outdated. It is based on lucene 3.0.x

Would Like to contribute to Lucene

-to-date resources. Can someone recommend a recently released book on lucene internals or has someone planned to write one? What would be the starting point if I need to learn the internals of Lucene? Thanks, -- Gimantha Bandara Software Engineer WSO2. Inc : http://wso2.com Mobile : +94714961919

Re: Sampled Hit counts using Lucene Facets.

on LongRangeFacetCounts (and Double), or a separate utility class which given min/max values and numBuckets, creates the proper Range[]? Shai On Tue, Mar 10, 2015 at 4:07 PM, Gimantha Bandara giman...@wso2.com wrote: Hi Shai, Yes, Splitting ranges into smaller ranges is not as same

Difference between StoredField vs Other Fields with Field.Store.YES

Hi all, Is there a difference between using StoredField and using other types of fields with Field.Store.YES? Another question, Is it a good practise to use NumericDocValuesField instead of using usual Fields (IntField, LongField, StringField ...etc) with Field.Store.NO ? -- Gimantha Bandara

Re: Sampled Hit counts using Lucene Facets.

implemented facet sampling. Shai On Tue, Mar 10, 2015 at 10:21 AM, Gimantha Bandara giman...@wso2.com wrote: What I am planning to do is, split the given time range into smaller time ranges by myself and pass them to a LongRangeFacetsCount object and get the counts for each sub range

Re: Lucene index

synonym in lucene PLEASE Also, is lucene index graph database or not? Thank U -- Gimantha Bandara Software Engineer WSO2. Inc : http://wso2.com Mobile : +94714961919

Re: Sampled Hit counts using Lucene Facets.

What I am planning to do is, split the given time range into smaller time ranges by myself and pass them to a LongRangeFacetsCount object and get the counts for each sub range. Is this the correct way? On Tue, Mar 10, 2015 at 12:01 AM, Gimantha Bandara giman...@wso2.com wrote: Any updates

Re: Sampled Hit counts using Lucene Facets.

Any updates on this please? Do I have to write my own code to sample and get the hitcount? On Sat, Mar 7, 2015 at 2:14 PM, Gimantha Bandara giman...@wso2.com wrote: Any help on this please? On Fri, Mar 6, 2015 at 3:13 PM, Gimantha Bandara giman...@wso2.com wrote: Hi, I am trying

Re: Sampled Hit counts using Lucene Facets.

Any help on this please? On Fri, Mar 6, 2015 at 3:13 PM, Gimantha Bandara giman...@wso2.com wrote: Hi, I am trying to create some APIs using lucene facets APIs. First I will explain my requirement with an example. Lets say I am keeping track of the count of people who enter through

Sampled Hit counts using Lucene Facets.

guide for lucene 4.10.3 or lucene 5.0.0 ? Thanks, -- Gimantha Bandara Software Engineer WSO2. Inc : http://wso2.com Mobile : +94714961919

Re: Difference between CategoryPath and Plain FacetFields with hierarchy

)) On Thu, Mar 5, 2015 at 8:17 PM, Michael McCandless luc...@mikemccandless.com wrote: The facets API changed in 5.0, from CategoryPath/FacetSearchParams to FacetFields/FacetsConfig. Mike McCandless http://blog.mikemccandless.com On Thu, Mar 5, 2015 at 2:13 AM, Gimantha Bandara giman

Re: Difference between CategoryPath and Plain FacetFields with hierarchy

Hi, Any help on this? Or Can someone point me to Faceted User guide of 4.10.3. I cannot find it. Is it only available for Older version? On Wed, Mar 4, 2015 at 2:38 PM, Gimantha Bandara giman...@wso2.com wrote: Hi, I am new to Lucene faceting and taxonomy. I saw few examples in some blogs

Difference between CategoryPath and Plain FacetFields with hierarchy

both these approaches? What is the recommended approach to to create a faceted search application? For so called two different approaches please refer to [1] and [2] [1] http://www.hascode.com/2012/08/lucene-snippets-faceting-search/ [2] http://www.norconex.com/facets-with-lucene/ -- Gimantha