Re: Update Field on Many Docs
Let me revise my question. After reading through the source code for Lucene a bit more, I realize that using updateDocValues isn't really an option. This leaves me with querying all the data, modifying each document, and replacing the original. Not as efficient as I was hoping for, but sufficient. This does work for me, but with one major flaw. Facets are lost because they aren't loaded into the Document when I search. Is there any way to retrieve the existing facet fields into the document before I write it back out? On Tue, May 8, 2018 9:02 AM, Matt Hicks m...@outr.com wrote: I'm trying to write an efficient update to match with a Query and update all matches with specific fields (while maintaining the other previously set fields on each document). I see IndexWriter.updateDocValues appears to be the closest solution, but it takes a Term and I'd prefer to use a Query. Is there any way to accomplish this in Lucene without querying and updating each entry manually?
Update Field on Many Docs
I'm trying to write an efficient update to match with a Query and update all matches with specific fields (while maintaining the other previously set fields on each document). I see IndexWriter.updateDocValues appears to be the closest solution, but it takes a Term and I'd prefer to use a Query. Is there any way to accomplish this in Lucene without querying and updating each entry manually?
Name Fuzzy Search
I'm attempting to write a Lucene query to match names giving me the ideal results first. I presume I should limit it to a minimum of two word matches and use fuzzy matching. Is there an ideal way to support this? Should I explicitly have name aliases (like Bob/Robert/Rob) or should I leave that to fuzzy matching? I am processing over millions of records and want to get the most likely match based on name. Thanks
Synchronized Commit?
When I add a document to my index I immediately commit (indexWriter.commit(), taxonomyWriter.commit(), and searcherTaxonomyManager.maybeRefresh()) and then when I do a search I call searcherTaxonomyManager.maybeRefreshBlocking() and then searcherTaxonomyManager.acquire() followed by a search, the new document is often not found yet. If I wait for a second and then do a search it finds it fine. Is there any way to force Lucene to synchronously commit or synchronously wait for the index to be fully up-to-date before doing a search?
Re: Facet DrillDown Exclusion
Thanks, that did the trick!
On Tue, Dec 6, 2016 at 8:58 AM Shai Erera <ser...@gmail.com> wrote:
> Hey Matt,
>
> You basically don't need to use DDQ in that case. You can construct a
> BooleanQuery with a MUST_NOT clause for filter out the facet path. Here's a
> short code snippet:
>
> String indexedField = config.getDimConfig("Author").indexFieldName; // Find
> the field of the "Author" facet
> Query q = new BooleanQuery.Builder()
> .add(new MatchAllDocsQuery(), Occur.MUST) // here you would usually use a
> different query
> .add(new TermQuery(DrillDownQuery.term(indexedField, "Author", "Lisa")),
> Occur.MUST_NOT) // do not match documents with "Author/Lisa" in their
> facets
> .build();
> searcher.search(q, 10);
>
> Hope this helps
>
> Shai
>
> On Tue, Dec 6, 2016 at 1:55 AM Matt Hicks <m...@outr.com> wrote:
>
> > I'm currently drilling down adding a facet path, but I'd like to be able
> to
> > do the same as a NOT query. Is there any way to do an exclusion drill
> down
> > on a facet to exclude docs that match the facet while including all
> others?
> >
> > Thanks
> >
>
Facet DrillDown Exclusion
I'm currently drilling down adding a facet path, but I'd like to be able to do the same as a NOT query. Is there any way to do an exclusion drill down on a facet to exclude docs that match the facet while including all others? Thanks
Re: LeafCollector
Interesting. Are there any examples of how to actually use DiversifiedTopDocsCollector? On Fri, Dec 2, 2016 at 8:53 AM Adrien Grand <jpou...@gmail.com> wrote: > Maybe you could use DiversifiedTopDocsCollector? > https://lucene.apache.org/core/6_2_0/misc/org/apache/lucene/search/DiversifiedTopDocsCollector.html > > Le jeu. 1 déc. 2016 à 23:08, Michael McCandless <luc...@mikemccandless.com> > a écrit : > > Lucene used to have a DuplicateFilter to do this, but we removed it > recently ... see https://issues.apache.org/jira/browse/LUCENE-6633 for > some discussion as to why. > > Mike McCandless > > http://blog.mikemccandless.com > > > On Thu, Dec 1, 2016 at 2:39 PM, Matt Hicks <m...@outr.com> wrote: > > I'm trying to write a LeafCollector that filters out duplicates for a > > specific field. However, looking at the JavaDoc for `collect` it says not > > to call `IndexSearch.doc` or `IndexReader.document`. How am I supposed > to > > determine the value of a field and then exclude it? > > - > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > >
LeafCollector
I'm trying to write a LeafCollector that filters out duplicates for a specific field. However, looking at the JavaDoc for `collect` it says not to call `IndexSearch.doc` or `IndexReader.document`. How am I supposed to determine the value of a field and then exclude it?
Re: StartsWith on DrillDown?
I understand this, and that's how I'm using it now, but my situation is
that in my application I want to offer the ability to auto-complete tags
that have results based on the current query. This is why I'm looking for
a "StartsWith" filter on the tags. Certainly I could get back all of the
tags and then filter them myself, but eventually there could be hundreds of
thousands of tags that I'm filtering through and if the user starts typing
"but" I want to be able to show "butterfly" if there are tag matches within
the current query. I'm currently using taxonomy facets.
On Thu, Nov 17, 2016 at 4:23 AM Michael McCandless <
luc...@mikemccandless.com> wrote:
> The idea w/ drill down is you are running a "base query" (what the
> user actually searched for, originally) and then, if the user has
> clicked to drill down on any facet labels, you are also adding
> drill-down queries.
>
> You pass the "base query" to the DrillDownQuery constructor.
>
> And, normally, to add drill-down queries, you would use the add method
> that takes only strings, when the user clicked on a dimension + label.
>
> The add method that takes a custom drill-down query is for more
> advanced use cases, where you are able to create your own query that
> accomplishes the same thing as drilling down by a label; e.g., for
> numeric range facets, you would use this method to pass a numeric
> range filter down.
>
> Have you seen the demo facet examples, e.g.
>
> https://github.com/apache/lucene-solr/blob/master/lucene/demo/src/java/org/apache/lucene/demo/facet/SimpleSortedSetFacetsExample.java
> ?
>
> Are you using SSDV facets or taxonomy facets?
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Wed, Nov 16, 2016 at 5:29 PM, Matt Hicks <m...@outr.com> wrote:
> > My situation is that I simply don't understand how I'm supposed to pass a
> > `Query` into it. Just passing in a `new QueryParser(facetName,
> > standardAnalyzer)` to `drillDown.add(facetName, queryParser.parse("valid
> > query"))` just always returns zero items. I'm guessing it has to do with
> > needing a specific type of query?
> >
> > On Wed, Nov 16, 2016 at 4:05 PM Michael McCandless
> > <luc...@mikemccandless.com> wrote:
> >>
> >> Can you post a test case showing the unexpected behavior?
> >>
> >> Mike McCandless
> >>
> >> http://blog.mikemccandless.com
> >>
> >> On Wed, Nov 16, 2016 at 1:55 PM, Matt Hicks <m...@outr.com> wrote:
> >> > Is this simply not possible to accomplish or does nobody on this list
> >> > know?
> >> >
> >> > On Mon, Nov 14, 2016 at 2:39 PM Matt Hicks <m...@outr.com> wrote:
> >> >
> >> >> I'm trying to add a sub-query to my DrillDownQuery but I keep ending
> up
> >> >> with no results when I use add(String dim, Query subQuery). I'm
> trying
> >> >> to
> >> >> query the tags that start with a specific String.
> >> >>
> >> >> Any suggestions of how to do this would be greatly appreciated. I am
> >> >> using
> >> >> Lucene Core 6.3.0.
> >> >>
> >> >> Thank you
> >> >>
>
Re: StartsWith on DrillDown?
My situation is that I simply don't understand how I'm supposed to pass a
`Query` into it. Just passing in a `new QueryParser(facetName,
standardAnalyzer)` to `drillDown.add(facetName, queryParser.parse("valid
query"))` just always returns zero items. I'm guessing it has to do with
needing a specific type of query?
On Wed, Nov 16, 2016 at 4:05 PM Michael McCandless <
luc...@mikemccandless.com> wrote:
> Can you post a test case showing the unexpected behavior?
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Wed, Nov 16, 2016 at 1:55 PM, Matt Hicks <m...@outr.com> wrote:
> > Is this simply not possible to accomplish or does nobody on this list
> know?
> >
> > On Mon, Nov 14, 2016 at 2:39 PM Matt Hicks <m...@outr.com> wrote:
> >
> >> I'm trying to add a sub-query to my DrillDownQuery but I keep ending up
> >> with no results when I use add(String dim, Query subQuery). I'm trying
> to
> >> query the tags that start with a specific String.
> >>
> >> Any suggestions of how to do this would be greatly appreciated. I am
> using
> >> Lucene Core 6.3.0.
> >>
> >> Thank you
> >>
>
Re: StartsWith on DrillDown?
Is this simply not possible to accomplish or does nobody on this list know? On Mon, Nov 14, 2016 at 2:39 PM Matt Hicks <m...@outr.com> wrote: > I'm trying to add a sub-query to my DrillDownQuery but I keep ending up > with no results when I use add(String dim, Query subQuery). I'm trying to > query the tags that start with a specific String. > > Any suggestions of how to do this would be greatly appreciated. I am using > Lucene Core 6.3.0. > > Thank you >
StartsWith on DrillDown?
I'm trying to add a sub-query to my DrillDownQuery but I keep ending up with no results when I use add(String dim, Query subQuery). I'm trying to query the tags that start with a specific String. Any suggestions of how to do this would be greatly appreciated. I am using Lucene Core 6.3.0. Thank you
