[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15825503#comment-15825503
 ] 

David Smiley commented on SOLR-9708:


As I was working on highlighter documentation... I think the default we use for 
hl.maxAnalyzedChars should be equal to that of the Original Highlighter, which 
is 51200 -- certainly not less.  After all, it certainly does a faster job, and 
this parameter is a performance oriented threshold.  I see we're currently 
using the default in the UnifiedHighlighter which is 1. This fits within 
the overarching goal of making transitioning to this highlighter 
straight-forward minimizing gotchas.

[~jim.ferenczi] can I simply commit a 1-liner change to set this default in 6.4 
or shall I file a new issue?

> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
> Attachments: SOLR-9708.patch
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15694497#comment-15694497
 ] 

ASF subversion and git services commented on SOLR-9708:
---

Commit a089bf2827e536b8894c4181844bcf07cb6f9c79 in lucene-solr's branch 
refs/heads/branch_6x from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a089bf2 ]

SOLR-9708: Added UnifiedSolrHighlighter.  Added 
hl.method=original|fastVector|postings|unified

(cherry picked from commit 4314c51)


> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
> Attachments: SOLR-9708.patch
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15694494#comment-15694494
 ] 

ASF subversion and git services commented on SOLR-9708:
---

Commit 4314c51c66de1eed0dbc4897684e79935ebfd55e in lucene-solr's branch 
refs/heads/master from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4314c51 ]

SOLR-9708: Added UnifiedSolrHighlighter.  Added 
hl.method=original|fastVector|postings|unified


> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
> Attachments: SOLR-9708.patch
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[Timothy M. Rodriguez (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15693717#comment-15693717
 ] 

Timothy M. Rodriguez commented on SOLR-9708:


Haha, no problem.  It'll improve usability quite a bit to be able to 
dynamically invoke it per request (and the other highlighters).  I'm glad it 
landed with the initial Solr release of the unified highlighter.

> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
> Attachments: SOLR-9708.patch
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[Timothy M. Rodriguez (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15691807#comment-15691807
 ] 

Timothy M. Rodriguez commented on SOLR-9708:


Looks great!  Adding the other highlighters to method really fleshed it out.  
Also in favor of the change from "default" to "original".  No further suggested 
changes other than a rename on the FASTVECTOR enum to FAST_VECTOR. Let me know 
if you need any help with the wiki in December.  Would be glad to contribute 
there as well.

> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15689027#comment-15689027
 ] 

David Smiley commented on SOLR-9708:


I iterated further from your work: 
https://github.com/dsmiley/lucene-solr/commit/a29dae9dde1dc018bd704cefb4fd7560d4eeb9fc
I expanded the support of {{hl.method}} to all of: 
original|fastVector|postings|unified.  I also organized/documented 
HighlightParams with notes on which highlighters use which.  I figure 
"original" is a nice name for the Highlighter.java implementation instead of 
"standard", as "original" will remain a good name, even if it's not necessarily 
the default in the future.  "standard" kinda means "default".

What do you think?  Feel free to either comment here or on my commit in GitHub; 
whatever suits you.  Once there's no further changes, I'll promptly post a 
final .patch file here then commit.

Later in December I'll update the Solr Ref Guide.  I might convert the 
PostingsHighlighter page to be about the UnifiedHighlighter, and then on this 
page address how the PostingsHighlighter varies and how it's invoked instead 
(very easy now thanks to hl.method=postings).  Since the UH is a derivative of 
the PH, completely compatible; we might very well remove the PH in 7.0.

> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15688233#comment-15688233
 ] 

ASF GitHub Bot commented on SOLR-9708:
--

Github user Timothy055 commented on the issue:

https://github.com/apache/lucene-solr/pull/107
  
I've made some more updates to the documentation for the pre and post 
parameters as well as fixed the defaulting logic.


> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15688177#comment-15688177
 ] 

ASF GitHub Bot commented on SOLR-9708:
--

Github user Timothy055 commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/107#discussion_r89227253
  
--- Diff: 
solr/core/src/java/org/apache/solr/handler/component/HighlightComponent.java ---
@@ -184,6 +185,20 @@ public void process(ResponseBuilder rb) throws 
IOException {
 }
   }
 
+  /**
+   * Normalizes parameters between highlighters
+   */
+  private SolrParams normalizeParameters(SolrParams params) {
--- End diff --

Fixed


> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15685594#comment-15685594
 ] 

ASF GitHub Bot commented on SOLR-9708:
--

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/107#discussion_r89039004
  
--- Diff: 
solr/core/src/java/org/apache/solr/handler/component/HighlightComponent.java ---
@@ -184,6 +185,20 @@ public void process(ResponseBuilder rb) throws 
IOException {
 }
   }
 
+  /**
+   * Normalizes parameters between highlighters
+   */
+  private SolrParams normalizeParameters(SolrParams params) {
--- End diff --

You've coded this such that SIMPLE_PRE overrides TAG_PRE which is not what 
we want I think?  Furthermore, this is coded such that it only overrides at the 
global level which won't work for field-specific settings like 
`f.myfieldname.hl.tag.pre` which we'd want to examine 
`f.myfieldname.hl.simple.pre`.  I appreciate where you were going with this, 
but in light of the latter point, I think you should simply modify the Solr UH 
adapter to lookup say "pre" like so:

String preTag = params.getFieldParam(fieldName, 
HighlightParams.TAG_PRE, 
   params.getFieldParam(fieldName, HighlightParams.SIMPLE_PRE, 
"");
 );



> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[Timothy M. Rodriguez (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15683730#comment-15683730
 ] 

Timothy M. Rodriguez commented on SOLR-9708:


Added a normalizeParameters method that will set tag.pre or post if simple.pre 
or post are set.

> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[Timothy M. Rodriguez (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15683698#comment-15683698
 ] 

Timothy M. Rodriguez commented on SOLR-9708:


I've posted an initial commit that allows the user to override the configured 
highlighter based on the "hl.method" parameter.

Two things I want to highlight:

* The highlighter can no longer safely be statically determined using 
HighlightComponent.getHiglighter since a request parameter can override the 
pre-configured one.  I've marked this usage deprecated as it affects quite a 
few places outside of this change.  Is that okay?

* Use of an enum for collecting all the highlight methods and giving a bit 
extra type safety when switching over the values in the override.  I'm not sure 
if this is out of style and several static String fields is preferred (although 
I personally prefer the former).

> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[Timothy M. Rodriguez (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668824#comment-15668824
 ] 

Timothy M. Rodriguez commented on SOLR-9708:


I was suggesting instead of hl.tag.pre, but realized that's used too. No sense 
adding a third. Even though both names are not so ideal IMO

> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668771#comment-15668771
 ] 

David Smiley commented on SOLR-9708:


I suggested to support _both_. 

> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[Timothy M. Rodriguez (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668766#comment-15668766
 ] 

Timothy M. Rodriguez commented on SOLR-9708:


I thought the suggestion was to use hl.tag.pre instead of hl.simple.pre?

> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668761#comment-15668761
 ] 

David Smiley commented on SOLR-9708:


-1 I would hate to see new parameters when there are semantically equivalent 
ones already. 

> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[Timothy M. Rodriguez (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668715#comment-15668715
 ] 

Timothy M. Rodriguez commented on SOLR-9708:


I'm okay with hl.tag.pre/post, but it may not always be a tag.  Perhaps 
something like hl.pre.marker? or hl.pre.sigil?

> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668334#comment-15668334
 ] 

David Smiley commented on SOLR-9708:


Also I'm on the fence if we should support {{hl.simple.pre}} 
({{HighlightParams.SIMPLE_PRE}}) and corresponding -post.  We probably should 
for a better user experience.  But support those as fallbacks, as I think the 
{{hl.tag.pre}} is a better name.

> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668291#comment-15668291
 ] 

David Smiley commented on SOLR-9708:


After you add hl.useUnifiedHighlighter, I think the tests should be updated to 
go about it his way instead of custom search component because people probably 
won't bother to explicitly change the search component given this approach 
works without fuss.

Maybe hl.method should be hl.offsetSource so as to not suggest you're picking 
the highlighter implementation overall?

> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15665417#comment-15665417
 ] 

David Smiley commented on SOLR-9708:


Sure, put the proposed logic into the HighlightComponent. 

It's definitely a separate issue for the Standard & FVH to be fully 
configurable via params. 

> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[Timothy M. Rodriguez (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15665291#comment-15665291
 ] 

Timothy M. Rodriguez commented on SOLR-9708:


Thanks for catching those things. I've fixed them and pushed to the pr.

Regarding the hl.useUnifiedHighlighter I'm actually very in favor of that idea, 
but perhaps that logic would be better in the highlight component?  In that way 
the actual highlighters would be more like the facet params that help tweak 
with algorithm gets used.  I agree that we really shouldn't have to "configure" 
the highlighters.  Perhaps that should be a separate issue though more in line 
with the the other changes mentioned?

> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15662936#comment-15662936
 ] 

David Smiley commented on SOLR-9708:


_(see github comments)_
Looking good Tim; just some super minor stuff.

It's unfortunate that users need to go into their solrconfig to enable this 
highlighter.  What if there was a boolean param {{hl.useUnifiedHighlighter}} 
that was evaluated at the very beginning of 
{{DefaultSolrHighlighter.doHighlighting}} that delegated to it?  I know from a 
code perspective it's _very_ hacky but consider the user experience.  In Solr 
we're trying to push towards avoiding the need to touch configs.  Fortunately, 
this highlighter is completely configured from params (not so for SH & FVH)... 
except for enabling it in the first place (which we can change here). The DSH 
after all already handles FVH.  In a future issue for 7.0 I'd like to propose 
some re-arrangement to the Solr highlighting code to simplify & clean up things.

> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15662927#comment-15662927
 ] 

ASF GitHub Bot commented on SOLR-9708:
--

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/107#discussion_r87741032
  
--- Diff: 
solr/core/src/java/org/apache/solr/highlight/UnifiedSolrHighlighter.java ---
@@ -0,0 +1,366 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.highlight;
+
+import java.io.IOException;
+import java.text.BreakIterator;
+import java.util.Collections;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.lucene.document.Document;
+import org.apache.lucene.search.DocIdSetIterator;
+import org.apache.lucene.search.Query;
+import org.apache.lucene.search.postingshighlight.WholeBreakIterator;
+import org.apache.lucene.search.uhighlight.DefaultPassageFormatter;
+import org.apache.lucene.search.uhighlight.PassageFormatter;
+import org.apache.lucene.search.uhighlight.PassageScorer;
+import org.apache.lucene.search.uhighlight.UnifiedHighlighter;
+import org.apache.solr.common.params.HighlightParams;
+import org.apache.solr.common.params.SolrParams;
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.common.util.SimpleOrderedMap;
+import org.apache.solr.core.PluginInfo;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.request.SolrRequestInfo;
+import org.apache.solr.schema.IndexSchema;
+import org.apache.solr.schema.SchemaField;
+import org.apache.solr.search.DocIterator;
+import org.apache.solr.search.DocList;
+import org.apache.solr.search.SolrIndexSearcher;
+import org.apache.solr.util.RTimerTree;
+import org.apache.solr.util.plugin.PluginInfoInitialized;
+
+/*
+ * TODO: The HighlightComponent should not call rewrite on the query; it 
should be up to the
+ * SolrHighlighter to do if needed.  Furthermore this arrangement is odd 
-- why are these abstractions separate?
+ */
+
+/**
+ * Highlighter impl that uses {@link UnifiedHighlighter}
+ * 
+ * Example configuration with default values:
+ * 
+ * requestHandler name="standard" 
class="solr.StandardRequestHandler"
+ * lst name="defaults"
+ * int name="hl.snippets"1/int
+ * str name="hl.tag.pre"lt;emgt;/str
+ * str name="hl.tag.post"lt;/emgt;/str
+ * str name="hl.tag.ellipsis"... /str
+ * bool name="hl.defaultSummary"true/bool
+ * str name="hl.encoder"simple/str
+ * float name="hl.score.k1"1.2/float
+ * float name="hl.score.b"0.75/float
+ * float name="hl.score.pivot"87/float
+ * str name="hl.bs.language"/str
+ * str name="hl.bs.country"/str
+ * str name="hl.bs.variant"/str
+ * str name="hl.bs.type"SENTENCE/str
+ * int name="hl.maxAnalyzedChars"1/int
+ * bool name="hl.highlightMultiTerm"true/bool
+ * /lst
+ * /requestHandler
+ * 
+ * ...
+ * 
+ * searchComponent class="solr.HighlightComponent" name="highlight"
+ * highlighting 
class="org.apache.solr.highlight.UnifiedSolrHighlighter"/
+ * /searchComponent
+ * 
+ * 
+ * Notes:
+ * 
+ * hl.q (string) can specify the query
+ * hl.fl (string) specifies the field list.
+ * hl.snippets (int) specifies how many snippets to return.
+ * hl.tag.pre (string) specifies text which appears before a 
highlighted term.
+ * hl.tag.post (string) specifies text which appears after a 
highlighted term.
+ * hl.tag.ellipsis (string) specifies text which joins non-adjacent 
passages. The default is to retain each
+ * value in a list without joining them.
+ * hl.defaultSummary (bool) specifies if a field should have a default 
summary of the leading text.
+ * hl.encoder (string) can be 'html' (html escapes content) or 
'simple' (no escaping).
+ * hl.score.k1 (float) specifies bm25 

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15662931#comment-15662931
 ] 

ASF GitHub Bot commented on SOLR-9708:
--

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/107#discussion_r87740491
  
--- Diff: 
solr/core/src/test/org/apache/solr/highlight/TestUnifiedSolrHighlighter.java ---
@@ -0,0 +1,222 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.highlight;
+
+import org.apache.solr.SolrTestCaseJ4;
+import org.apache.solr.handler.component.HighlightComponent;
+import org.apache.solr.schema.IndexSchema;
+import org.junit.BeforeClass;
+import org.junit.Ignore;
+
+/** simple tests for PostingsSolrHighlighter */
+public class TestUnifiedSolrHighlighter extends SolrTestCaseJ4 {
+  
+  @BeforeClass
+  public static void beforeClass() throws Exception {
+initCore("solrconfig-unifiedhighlight.xml", 
"schema-unifiedhighlight.xml");
+
+// test our config is sane, just to be sure:
+
+// postingshighlighter should be used
+SolrHighlighter highlighter = 
HighlightComponent.getHighlighter(h.getCore());
+assertTrue("wrong highlighter: " + highlighter.getClass(), highlighter 
instanceof UnifiedSolrHighlighter);
+
+// 'text' and 'text3' should have offsets, 'text2' should not
+IndexSchema schema = h.getCore().getLatestSchema();
+assertTrue(schema.getField("text").storeOffsetsWithPositions());
+assertTrue(schema.getField("text3").storeOffsetsWithPositions());
+assertFalse(schema.getField("text2").storeOffsetsWithPositions());
+  }
+  
+  @Override
+  public void setUp() throws Exception {
+super.setUp();
+clearIndex();
+assertU(adoc("text", "document one", "text2", "document one", "text3", 
"crappy document", "id", "101"));
+assertU(adoc("text", "second document", "text2", "second document", 
"text3", "crappier document", "id", "102"));
+assertU(commit());
+  }
+  
+  public void testSimple() {
+assertQ("simplest test", 
+req("q", "text:document", "sort", "id asc", "hl", "true"),
+"count(//lst[@name='highlighting']/*)=2",
+
"//lst[@name='highlighting']/lst[@name='101']/arr[@name='text']/str='document
 one'",
+
"//lst[@name='highlighting']/lst[@name='102']/arr[@name='text']/str='second 
document'");
+  }
+
+  public void testMultipleSnippetsReturned() {
+clearIndex();
+assertU(adoc("text", "Document snippet one. Intermediate sentence. 
Document snippet two.",
+"text2", "document one", "text3", "crappy document", "id", "101"));
+assertU(commit());
+assertQ("multiple snippets test",
+req("q", "text:document", "sort", "id asc", "hl", "true", 
"hl.snippets", "2", "hl.bs.type", "SENTENCE"),
+
"count(//lst[@name='highlighting']/lst[@name='101']/arr[@name='text']/*)=2",
+
"//lst[@name='highlighting']/lst[@name='101']/arr/str[1]='Document 
snippet one. '",
+
"//lst[@name='highlighting']/lst[@name='101']/arr/str[2]='Document 
snippet two.'");
+  }
+
+  public void testStrictPhrasesEnabledByDefault() {
+clearIndex();
+assertU(adoc("text", "Strict phrases should be enabled for phrases",
+"text2", "document one", "text3", "crappy document", "id", "101"));
+assertU(commit());
+assertQ("strict phrase handling",
+req("q", "text:\"strict phrases\"", "sort", "id asc", "hl", 
"true"),
+
"count(//lst[@name='highlighting']/lst[@name='101']/arr[@name='text']/*)=1",
+
"//lst[@name='highlighting']/lst[@name='101']/arr/str[1]='Strict 
phrases should be enabled for phrases'");
+  }
+
+  public void testStrictPhrasesCanBeDisabled() {
+clearIndex();
+assertU(adoc("text", "Strict phrases should be disabled for phrases",
+

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15662930#comment-15662930
 ] 

ASF GitHub Bot commented on SOLR-9708:
--

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/107#discussion_r87738726
  
--- Diff: 
solr/core/src/java/org/apache/solr/highlight/UnifiedSolrHighlighter.java ---
@@ -0,0 +1,366 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.highlight;
+
+import java.io.IOException;
+import java.text.BreakIterator;
+import java.util.Collections;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.lucene.document.Document;
+import org.apache.lucene.search.DocIdSetIterator;
+import org.apache.lucene.search.Query;
+import org.apache.lucene.search.postingshighlight.WholeBreakIterator;
+import org.apache.lucene.search.uhighlight.DefaultPassageFormatter;
+import org.apache.lucene.search.uhighlight.PassageFormatter;
+import org.apache.lucene.search.uhighlight.PassageScorer;
+import org.apache.lucene.search.uhighlight.UnifiedHighlighter;
+import org.apache.solr.common.params.HighlightParams;
+import org.apache.solr.common.params.SolrParams;
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.common.util.SimpleOrderedMap;
+import org.apache.solr.core.PluginInfo;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.request.SolrRequestInfo;
+import org.apache.solr.schema.IndexSchema;
+import org.apache.solr.schema.SchemaField;
+import org.apache.solr.search.DocIterator;
+import org.apache.solr.search.DocList;
+import org.apache.solr.search.SolrIndexSearcher;
+import org.apache.solr.util.RTimerTree;
+import org.apache.solr.util.plugin.PluginInfoInitialized;
+
+/*
+ * TODO: The HighlightComponent should not call rewrite on the query; it 
should be up to the
+ * SolrHighlighter to do if needed.  Furthermore this arrangement is odd 
-- why are these abstractions separate?
+ */
+
+/**
+ * Highlighter impl that uses {@link UnifiedHighlighter}
+ * 
+ * Example configuration with default values:
+ * 
+ * requestHandler name="standard" 
class="solr.StandardRequestHandler"
+ * lst name="defaults"
+ * int name="hl.snippets"1/int
+ * str name="hl.tag.pre"lt;emgt;/str
+ * str name="hl.tag.post"lt;/emgt;/str
+ * str name="hl.tag.ellipsis"... /str
+ * bool name="hl.defaultSummary"true/bool
+ * str name="hl.encoder"simple/str
+ * float name="hl.score.k1"1.2/float
+ * float name="hl.score.b"0.75/float
+ * float name="hl.score.pivot"87/float
+ * str name="hl.bs.language"/str
+ * str name="hl.bs.country"/str
+ * str name="hl.bs.variant"/str
+ * str name="hl.bs.type"SENTENCE/str
+ * int name="hl.maxAnalyzedChars"1/int
+ * bool name="hl.highlightMultiTerm"true/bool
+ * /lst
+ * /requestHandler
+ * 
+ * ...
+ * 
+ * searchComponent class="solr.HighlightComponent" name="highlight"
+ * highlighting 
class="org.apache.solr.highlight.UnifiedSolrHighlighter"/
+ * /searchComponent
+ * 
+ * 
+ * Notes:
+ * 
+ * hl.q (string) can specify the query
+ * hl.fl (string) specifies the field list.
+ * hl.snippets (int) specifies how many snippets to return.
+ * hl.tag.pre (string) specifies text which appears before a 
highlighted term.
+ * hl.tag.post (string) specifies text which appears after a 
highlighted term.
+ * hl.tag.ellipsis (string) specifies text which joins non-adjacent 
passages. The default is to retain each
+ * value in a list without joining them.
+ * hl.defaultSummary (bool) specifies if a field should have a default 
summary of the leading text.
+ * hl.encoder (string) can be 'html' (html escapes content) or 
'simple' (no escaping).
+ * hl.score.k1 (float) specifies bm25 

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15662929#comment-15662929
 ] 

ASF GitHub Bot commented on SOLR-9708:
--

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/107#discussion_r87740728
  
--- Diff: 
solr/core/src/test/org/apache/solr/highlight/TestUnifiedSolrHighlighter.java ---
@@ -0,0 +1,222 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.highlight;
+
+import org.apache.solr.SolrTestCaseJ4;
+import org.apache.solr.handler.component.HighlightComponent;
+import org.apache.solr.schema.IndexSchema;
+import org.junit.BeforeClass;
+import org.junit.Ignore;
+
+/** simple tests for PostingsSolrHighlighter */
--- End diff --

obsolete reference to PostingsSolrHighlighter


> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15662925#comment-15662925
 ] 

ASF GitHub Bot commented on SOLR-9708:
--

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/107#discussion_r87740749
  
--- Diff: 
solr/core/src/test-files/solr/collection1/conf/solrconfig-unifiedhighlight.xml 
---
@@ -0,0 +1,35 @@
+
+
+
+
+
--- End diff --

obsolete postings highlighter reference


> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15662926#comment-15662926
 ] 

ASF GitHub Bot commented on SOLR-9708:
--

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/107#discussion_r87743142
  
--- Diff: 
solr/core/src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java ---
@@ -373,6 +373,11 @@ protected BoundaryScanner getBoundaryScanner(String 
fieldName, SolrParams params
 if (!isHighlightingEnabled(params)) // also returns early if no unique 
key field
   return null;
 
+boolean rewrite = query != null && 
!(Boolean.valueOf(params.get(HighlightParams.USE_PHRASE_HIGHLIGHTER, "true")) &&
--- End diff --

no biggie but I think a simple if(...) condition would be simpler; no 
variable.


> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15662928#comment-15662928
 ] 

ASF GitHub Bot commented on SOLR-9708:
--

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/107#discussion_r87740550
  
--- Diff: 
solr/core/src/test/org/apache/solr/highlight/TestUnifiedSolrHighlighter.java ---
@@ -0,0 +1,222 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.highlight;
+
+import org.apache.solr.SolrTestCaseJ4;
+import org.apache.solr.handler.component.HighlightComponent;
+import org.apache.solr.schema.IndexSchema;
+import org.junit.BeforeClass;
+import org.junit.Ignore;
+
+/** simple tests for PostingsSolrHighlighter */
+public class TestUnifiedSolrHighlighter extends SolrTestCaseJ4 {
+  
+  @BeforeClass
+  public static void beforeClass() throws Exception {
+initCore("solrconfig-unifiedhighlight.xml", 
"schema-unifiedhighlight.xml");
+
+// test our config is sane, just to be sure:
+
+// postingshighlighter should be used
+SolrHighlighter highlighter = 
HighlightComponent.getHighlighter(h.getCore());
+assertTrue("wrong highlighter: " + highlighter.getClass(), highlighter 
instanceof UnifiedSolrHighlighter);
+
+// 'text' and 'text3' should have offsets, 'text2' should not
+IndexSchema schema = h.getCore().getLatestSchema();
+assertTrue(schema.getField("text").storeOffsetsWithPositions());
+assertTrue(schema.getField("text3").storeOffsetsWithPositions());
+assertFalse(schema.getField("text2").storeOffsetsWithPositions());
+  }
+  
+  @Override
+  public void setUp() throws Exception {
+super.setUp();
+clearIndex();
+assertU(adoc("text", "document one", "text2", "document one", "text3", 
"crappy document", "id", "101"));
+assertU(adoc("text", "second document", "text2", "second document", 
"text3", "crappier document", "id", "102"));
+assertU(commit());
+  }
+  
+  public void testSimple() {
+assertQ("simplest test", 
+req("q", "text:document", "sort", "id asc", "hl", "true"),
+"count(//lst[@name='highlighting']/*)=2",
+
"//lst[@name='highlighting']/lst[@name='101']/arr[@name='text']/str='document
 one'",
+
"//lst[@name='highlighting']/lst[@name='102']/arr[@name='text']/str='second 
document'");
+  }
+
+  public void testMultipleSnippetsReturned() {
+clearIndex();
+assertU(adoc("text", "Document snippet one. Intermediate sentence. 
Document snippet two.",
+"text2", "document one", "text3", "crappy document", "id", "101"));
+assertU(commit());
+assertQ("multiple snippets test",
+req("q", "text:document", "sort", "id asc", "hl", "true", 
"hl.snippets", "2", "hl.bs.type", "SENTENCE"),
+
"count(//lst[@name='highlighting']/lst[@name='101']/arr[@name='text']/*)=2",
+
"//lst[@name='highlighting']/lst[@name='101']/arr/str[1]='Document 
snippet one. '",
+
"//lst[@name='highlighting']/lst[@name='101']/arr/str[2]='Document 
snippet two.'");
+  }
+
+  public void testStrictPhrasesEnabledByDefault() {
+clearIndex();
+assertU(adoc("text", "Strict phrases should be enabled for phrases",
+"text2", "document one", "text3", "crappy document", "id", "101"));
+assertU(commit());
+assertQ("strict phrase handling",
+req("q", "text:\"strict phrases\"", "sort", "id asc", "hl", 
"true"),
+
"count(//lst[@name='highlighting']/lst[@name='101']/arr[@name='text']/*)=1",
+
"//lst[@name='highlighting']/lst[@name='101']/arr/str[1]='Strict 
phrases should be enabled for phrases'");
+  }
+
+  public void testStrictPhrasesCanBeDisabled() {
+clearIndex();
+assertU(adoc("text", "Strict phrases should be disabled for phrases",
+

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[Timothy M. Rodriguez (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15662087#comment-15662087
 ] 

Timothy M. Rodriguez commented on SOLR-9708:


Let me know what you think.  If it looks good, I think we can commit it.

> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[Timothy M. Rodriguez (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15662086#comment-15662086
 ] 

Timothy M. Rodriguez commented on SOLR-9708:


I've pushed tests for the configurable items in the UH as well as for support 
of multiple snippets.  In addition a change was done to push highlighter 
specific logic down into the DefaultSolrHighlighter that was in the 
HighlightComponent (thanks [~dsmiley] for pointing that out).

> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.4
>
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15623477#comment-15623477
 ] 

David Smiley commented on SOLR-9708:


To anyone following along: The code is the PostingsHighlighter's Solr adapter, 
with just a few changes to work with the UH instead.

> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Priority: Minor
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9708) Expose UnifiedHighlighter in Solr

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15623438#comment-15623438
 ] 

ASF GitHub Bot commented on SOLR-9708:
--

GitHub user Timothy055 opened a pull request:

https://github.com/apache/lucene-solr/pull/107

SOLR-9708 UnifiedHighlighter Solr Plugin

An initial implementation of a solr plugin for the unified highlighter.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Timothy055/lucene-solr features/SOLR-9708

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/lucene-solr/pull/107.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #107


commit 669f94f27c73b9fd5190a980a089cff9d4df3015
Author: Timothy Rodriguez 
Date:   2016-10-31T21:05:47Z

initial commit of UnifiedSolrHighlighter adapter

commit 2cf4b2d35f1f535e8b8126b5dd685dc3bb663391
Author: Timothy Rodriguez 
Date:   2016-10-31T21:12:19Z

add license header




> Expose UnifiedHighlighter in Solr
> -
>
> Key: SOLR-9708
> URL: https://issues.apache.org/jira/browse/SOLR-9708
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Timothy M. Rodriguez
>Priority: Minor
>
> This ticket is for creating a Solr plugin that can utilize the new 
> UnifiedHighlighter which was initially committed in 
> https://issues.apache.org/jira/browse/LUCENE-7438



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org