Re: CHANGES.txt updates for SOLR-1516 and SOLR-1592

[Mattmann, Chris A (388J)]

Thanks, Hoss, no problemo, appreciate it!


On 1/26/10 12:22 PM, Chris Hostetter hossman_luc...@fucit.org wrote:



: Not to be a best, but there's no CHANGES.txt updates for SOLR-1516 and
: SOLR-1592. Could someone update them? A trivial patch is attached...

Sorry about that.

Every change (with the possible exception of fixing formating or
documentation typos) *should* have a CHANGES.txt entry.

Every change that affects the public API *MUST* have a CHANGES.txt entry.

Committed revision 903398.


-Hoss




++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.mattm...@jpl.nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++

Re: CHANGES.txt updates for SOLR-1516 and SOLR-1592

[Chris Hostetter]

: Not to be a best, but there's no CHANGES.txt updates for SOLR-1516 and
: SOLR-1592. Could someone update them? A trivial patch is attached...

Sorry about that.

Every change (with the possible exception of fixing formating or 
documentation typos) *should* have a CHANGES.txt entry.

Every change that affects the public API *MUST* have a CHANGES.txt entry.

Committed revision 903398.


-Hoss

CHANGES.txt updates for SOLR-1516 and SOLR-1592

[Mattmann, Chris A (388J)]

Hi Guys,

Not to be a best, but there's no CHANGES.txt updates for SOLR-1516 and
SOLR-1592. Could someone update them? A trivial patch is attached...

Cheers,
Chris


++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.mattm...@jpl.nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++

Re: CHANGES.txt updates for SOLR-1516 and SOLR-1592

[Mattmann, Chris A (388J)]

Dang,

Mailing list stripped the attachment. Here's a link to one:

http://sunset.usc.edu/~mattmann/CHANGES-solr.patch

Cheers,
Chris

On 1/12/10 10:39 PM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov 
wrote:

Hi Guys,

Not to be a best, but there's no CHANGES.txt updates for SOLR-1516 and
SOLR-1592. Could someone update them? A trivial patch is attached...

Cheers,
Chris


++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.mattm...@jpl.nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++





++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.mattm...@jpl.nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++

Re: svn commit: r823705 - /lucene/solr/trunk/CHANGES.txt

[Yonik Seeley]

I don't envy casual users trying to make sense out of some of these
release notes... any suggestions on how to make it easier are welcome.

-Yonik
http://www.lucidimagination.com



On Fri, Oct 9, 2009 at 5:45 PM,  yo...@apache.org wrote:
 Author: yonik
 Date: Fri Oct  9 21:45:49 2009
 New Revision: 823705

 URL: http://svn.apache.org/viewvc?rev=823705view=rev
 Log:
 doc - per-segment fieldcache issues

 Modified:
    lucene/solr/trunk/CHANGES.txt

 Modified: lucene/solr/trunk/CHANGES.txt
 URL: 
 http://svn.apache.org/viewvc/lucene/solr/trunk/CHANGES.txt?rev=823705r1=823704r2=823705view=diff
 ==
 --- lucene/solr/trunk/CHANGES.txt (original)
 +++ lucene/solr/trunk/CHANGES.txt Fri Oct  9 21:45:49 2009
 @@ -30,6 +30,20 @@
  faster for most cases.  One can revert to the previous algorithm (which has
  also been improved somewhat) by adding facet.method=enum to the request.

 +Searching and sorting is now done on a per-segment basis, meaning that
 +the FieldCache entries used for sorting and for function queries are
 +created and used per-segment and can be reused for segments that don't
 +change between index updates.  While generally beneficial, this can lead
 +to increased memory usage over 1.3 in certain scenarios:
 +  1) A single valued field that was used for both sorting and faceting
 +in 1.3 would have used the same top level FieldCache entry.  In 1.4,
 +sorting will use entries at the segment level while faceting will still
 +use entries at the top reader level, leading to increased memory usage.
 +  2) Certain function queries such as ord() and rord() require a top level
 +FieldCache instance and can thus lead to increased memory usage.  Consider
 +replacing ord() and rord() with alternatives, such as function queries
 +based on ms() for date boosting.
 +
  If you use custom Tokenizer or TokenFilter components in a chain specified in
  schema.xml, they must support reusability.  If your Tokenizer or TokenFilter
  maintains state, it should implement reset().  If your TokenFilteFactory does

Re: svn commit: r819314 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java src/test/org/apache/solr/highlight/HighlighterTest.java

[Koji Sekiguchi]

 Also make both options default to true.

If so, isn't this line (from HighlightComponent) needed to be
also true by default?

   boolean rewrite = 
!(Boolean.valueOf(req.getParams().get(HighlightParams.USE_PHRASE_HIGHLIGHTER)) 
 
Boolean.valueOf(req.getParams().get(HighlightParams.HIGHLIGHT_MULTI_TERM)));


I think MultiTermQueries are converted to ConstantScoreQuery
by rewrite?

Koji


markrmil...@apache.org wrote:

Author: markrmiller
Date: Sun Sep 27 13:58:30 2009
New Revision: 819314

URL: http://svn.apache.org/viewvc?rev=819314view=rev
Log:
SOLR-1221: Change Solr Highlighting to use the SpanScorer with MultiTerm 
expansion by default

Modified:
lucene/solr/trunk/CHANGES.txt

lucene/solr/trunk/src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java
lucene/solr/trunk/src/test/org/apache/solr/highlight/HighlighterTest.java

Modified: lucene/solr/trunk/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/lucene/solr/trunk/CHANGES.txt?rev=819314r1=819313r2=819314view=diff
==
--- lucene/solr/trunk/CHANGES.txt (original)
+++ lucene/solr/trunk/CHANGES.txt Sun Sep 27 13:58:30 2009
@@ -503,8 +503,8 @@
 45. SOLR-1078: Fixes to WordDelimiterFilter to avoid splitting or dropping
 international non-letter characters such as non spacing marks. (yonik)
 
-46. SOLR-825: Enables highlighting for range/wildcard/fuzzy/prefix queries if using hl.usePhraseHighlighter=true

-and hl.highlightMultiTerm=true.  (Mark Miller)
+46. SOLR-825, SOLR-1221: Enables highlighting for range/wildcard/fuzzy/prefix 
queries if using hl.usePhraseHighlighter=true
+and hl.highlightMultiTerm=true. Also make both options default to true. 
(Mark Miller)
 
 47. SOLR-1174: Fix Logging admin form submit url for multicore. (Jacob Singh via shalin)
 


Modified: 
lucene/solr/trunk/src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java
URL: 
http://svn.apache.org/viewvc/lucene/solr/trunk/src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java?rev=819314r1=819313r2=819314view=diff
==
--- 
lucene/solr/trunk/src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java
 (original)
+++ 
lucene/solr/trunk/src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java
 Sun Sep 27 13:58:30 2009
@@ -144,7 +144,7 @@
*/
   private QueryScorer getSpanQueryScorer(Query query, String fieldName, 
TokenStream tokenStream, SolrQueryRequest request) throws IOException {
 boolean reqFieldMatch = request.getParams().getFieldBool(fieldName, 
HighlightParams.FIELD_MATCH, false);
-Boolean highlightMultiTerm = 
request.getParams().getBool(HighlightParams.HIGHLIGHT_MULTI_TERM);
+Boolean highlightMultiTerm = 
request.getParams().getBool(HighlightParams.HIGHLIGHT_MULTI_TERM, true);
 if(highlightMultiTerm == null) {
   highlightMultiTerm = false;
 }
@@ -306,8 +306,9 @@
 }
  
 Highlighter highlighter;

-if 
(Boolean.valueOf(req.getParams().get(HighlightParams.USE_PHRASE_HIGHLIGHTER))) {
-  // wrap CachingTokenFilter around TokenStream for reuse
+if 
(Boolean.valueOf(req.getParams().get(HighlightParams.USE_PHRASE_HIGHLIGHTER, 
true))) {
+  // TODO: this is not always necessary - eventually we would like 
to avoid this wrap
+  //   when it is not needed.
   tstream = new CachingTokenFilter(tstream);
   
   // get highlighter


Modified: 
lucene/solr/trunk/src/test/org/apache/solr/highlight/HighlighterTest.java
URL: 
http://svn.apache.org/viewvc/lucene/solr/trunk/src/test/org/apache/solr/highlight/HighlighterTest.java?rev=819314r1=819313r2=819314view=diff
==
--- lucene/solr/trunk/src/test/org/apache/solr/highlight/HighlighterTest.java 
(original)
+++ lucene/solr/trunk/src/test/org/apache/solr/highlight/HighlighterTest.java 
Sun Sep 27 13:58:30 2009
@@ -585,6 +585,7 @@
 args.put(hl.fl, t_text);
 args.put(hl.fragsize, 40);
 args.put(hl.snippets, 10);
+args.put(hl.usePhraseHighlighter, false);
 
 TestHarness.LocalRequestFactory sumLRF = h.getRequestFactory(

   standard, 0, 200, args);



  

Re: svn commit: r819314 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java src/test/org/apache/solr/highlight/HighlighterTest.java

[Mark Miller]

Thank you Koji!

Indeed - those need to default true now.

Nice eyes, and thanks for looking over!

Koji Sekiguchi wrote:
  Also make both options default to true.

 If so, isn't this line (from HighlightComponent) needed to be
 also true by default?

boolean rewrite =
 !(Boolean.valueOf(req.getParams().get(HighlightParams.USE_PHRASE_HIGHLIGHTER))
 
 Boolean.valueOf(req.getParams().get(HighlightParams.HIGHLIGHT_MULTI_TERM)));


 I think MultiTermQueries are converted to ConstantScoreQuery
 by rewrite?

 Koji


 markrmil...@apache.org wrote:
 Author: markrmiller
 Date: Sun Sep 27 13:58:30 2009
 New Revision: 819314

 URL: http://svn.apache.org/viewvc?rev=819314view=rev
 Log:
 SOLR-1221: Change Solr Highlighting to use the SpanScorer with
 MultiTerm expansion by default

 Modified:
 lucene/solr/trunk/CHANGES.txt

 lucene/solr/trunk/src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java


 lucene/solr/trunk/src/test/org/apache/solr/highlight/HighlighterTest.java


 Modified: lucene/solr/trunk/CHANGES.txt
 URL:
 http://svn.apache.org/viewvc/lucene/solr/trunk/CHANGES.txt?rev=819314r1=819313r2=819314view=diff

 ==

 --- lucene/solr/trunk/CHANGES.txt (original)
 +++ lucene/solr/trunk/CHANGES.txt Sun Sep 27 13:58:30 2009
 @@ -503,8 +503,8 @@
  45. SOLR-1078: Fixes to WordDelimiterFilter to avoid splitting or
 dropping
  international non-letter characters such as non spacing marks.
 (yonik)
  -46. SOLR-825: Enables highlighting for
 range/wildcard/fuzzy/prefix queries if using
 hl.usePhraseHighlighter=true
 -and hl.highlightMultiTerm=true.  (Mark Miller)
 +46. SOLR-825, SOLR-1221: Enables highlighting for
 range/wildcard/fuzzy/prefix queries if using
 hl.usePhraseHighlighter=true
 +and hl.highlightMultiTerm=true. Also make both options default
 to true. (Mark Miller)
  
  47. SOLR-1174: Fix Logging admin form submit url for multicore.
 (Jacob Singh via shalin)
  

 Modified:
 lucene/solr/trunk/src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java

 URL:
 http://svn.apache.org/viewvc/lucene/solr/trunk/src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java?rev=819314r1=819313r2=819314view=diff

 ==

 ---
 lucene/solr/trunk/src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java
 (original)
 +++
 lucene/solr/trunk/src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java
 Sun Sep 27 13:58:30 2009
 @@ -144,7 +144,7 @@
 */
private QueryScorer getSpanQueryScorer(Query query, String
 fieldName, TokenStream tokenStream, SolrQueryRequest request) throws
 IOException {
  boolean reqFieldMatch =
 request.getParams().getFieldBool(fieldName,
 HighlightParams.FIELD_MATCH, false);
 -Boolean highlightMultiTerm =
 request.getParams().getBool(HighlightParams.HIGHLIGHT_MULTI_TERM);
 +Boolean highlightMultiTerm =
 request.getParams().getBool(HighlightParams.HIGHLIGHT_MULTI_TERM, true);
  if(highlightMultiTerm == null) {
highlightMultiTerm = false;
  }
 @@ -306,8 +306,9 @@
  }
Highlighter highlighter;
 -if
 (Boolean.valueOf(req.getParams().get(HighlightParams.USE_PHRASE_HIGHLIGHTER)))
 {
 -  // wrap CachingTokenFilter around TokenStream for reuse
 +if
 (Boolean.valueOf(req.getParams().get(HighlightParams.USE_PHRASE_HIGHLIGHTER,
 true))) {
 +  // TODO: this is not always necessary - eventually we
 would like to avoid this wrap
 +  //   when it is not needed.
tstream = new CachingTokenFilter(tstream);
   // get highlighter

 Modified:
 lucene/solr/trunk/src/test/org/apache/solr/highlight/HighlighterTest.java

 URL:
 http://svn.apache.org/viewvc/lucene/solr/trunk/src/test/org/apache/solr/highlight/HighlighterTest.java?rev=819314r1=819313r2=819314view=diff

 ==

 ---
 lucene/solr/trunk/src/test/org/apache/solr/highlight/HighlighterTest.java
 (original)
 +++
 lucene/solr/trunk/src/test/org/apache/solr/highlight/HighlighterTest.java
 Sun Sep 27 13:58:30 2009
 @@ -585,6 +585,7 @@
  args.put(hl.fl, t_text);
  args.put(hl.fragsize, 40);
  args.put(hl.snippets, 10);
 +args.put(hl.usePhraseHighlighter, false);
  
  TestHarness.LocalRequestFactory sumLRF = h.getRequestFactory(
standard, 0, 200, args);



   



-- 
- Mark

http://www.lucidimagination.com

Re: svn commit: r819314 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java src/test/org/apache/solr/highlight/HighlighterTest.java

[Mark Miller]

Turned up a bug to boot ...

Mark Miller wrote:
 Thank you Koji!

 Indeed - those need to default true now.

 Nice eyes, and thanks for looking over!

 Koji Sekiguchi wrote:
   
 Also make both options default to true.
   
 If so, isn't this line (from HighlightComponent) needed to be
 also true by default?

boolean rewrite =
 !(Boolean.valueOf(req.getParams().get(HighlightParams.USE_PHRASE_HIGHLIGHTER))
 
 Boolean.valueOf(req.getParams().get(HighlightParams.HIGHLIGHT_MULTI_TERM)));


 I think MultiTermQueries are converted to ConstantScoreQuery
 by rewrite?

 Koji


 markrmil...@apache.org wrote:
 
 Author: markrmiller
 Date: Sun Sep 27 13:58:30 2009
 New Revision: 819314

 URL: http://svn.apache.org/viewvc?rev=819314view=rev
 Log:
 SOLR-1221: Change Solr Highlighting to use the SpanScorer with
 MultiTerm expansion by default

 Modified:
 lucene/solr/trunk/CHANGES.txt

 lucene/solr/trunk/src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java


 lucene/solr/trunk/src/test/org/apache/solr/highlight/HighlighterTest.java


 Modified: lucene/solr/trunk/CHANGES.txt
 URL:
 http://svn.apache.org/viewvc/lucene/solr/trunk/CHANGES.txt?rev=819314r1=819313r2=819314view=diff

 ==

 --- lucene/solr/trunk/CHANGES.txt (original)
 +++ lucene/solr/trunk/CHANGES.txt Sun Sep 27 13:58:30 2009
 @@ -503,8 +503,8 @@
  45. SOLR-1078: Fixes to WordDelimiterFilter to avoid splitting or
 dropping
  international non-letter characters such as non spacing marks.
 (yonik)
  -46. SOLR-825: Enables highlighting for
 range/wildcard/fuzzy/prefix queries if using
 hl.usePhraseHighlighter=true
 -and hl.highlightMultiTerm=true.  (Mark Miller)
 +46. SOLR-825, SOLR-1221: Enables highlighting for
 range/wildcard/fuzzy/prefix queries if using
 hl.usePhraseHighlighter=true
 +and hl.highlightMultiTerm=true. Also make both options default
 to true. (Mark Miller)
  
  47. SOLR-1174: Fix Logging admin form submit url for multicore.
 (Jacob Singh via shalin)
  

 Modified:
 lucene/solr/trunk/src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java

 URL:
 http://svn.apache.org/viewvc/lucene/solr/trunk/src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java?rev=819314r1=819313r2=819314view=diff

 ==

 ---
 lucene/solr/trunk/src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java
 (original)
 +++
 lucene/solr/trunk/src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java
 Sun Sep 27 13:58:30 2009
 @@ -144,7 +144,7 @@
 */
private QueryScorer getSpanQueryScorer(Query query, String
 fieldName, TokenStream tokenStream, SolrQueryRequest request) throws
 IOException {
  boolean reqFieldMatch =
 request.getParams().getFieldBool(fieldName,
 HighlightParams.FIELD_MATCH, false);
 -Boolean highlightMultiTerm =
 request.getParams().getBool(HighlightParams.HIGHLIGHT_MULTI_TERM);
 +Boolean highlightMultiTerm =
 request.getParams().getBool(HighlightParams.HIGHLIGHT_MULTI_TERM, true);
  if(highlightMultiTerm == null) {
highlightMultiTerm = false;
  }
 @@ -306,8 +306,9 @@
  }
Highlighter highlighter;
 -if
 (Boolean.valueOf(req.getParams().get(HighlightParams.USE_PHRASE_HIGHLIGHTER)))
 {
 -  // wrap CachingTokenFilter around TokenStream for reuse
 +if
 (Boolean.valueOf(req.getParams().get(HighlightParams.USE_PHRASE_HIGHLIGHTER,
 true))) {
 +  // TODO: this is not always necessary - eventually we
 would like to avoid this wrap
 +  //   when it is not needed.
tstream = new CachingTokenFilter(tstream);
   // get highlighter

 Modified:
 lucene/solr/trunk/src/test/org/apache/solr/highlight/HighlighterTest.java

 URL:
 http://svn.apache.org/viewvc/lucene/solr/trunk/src/test/org/apache/solr/highlight/HighlighterTest.java?rev=819314r1=819313r2=819314view=diff

 ==

 ---
 lucene/solr/trunk/src/test/org/apache/solr/highlight/HighlighterTest.java
 (original)
 +++
 lucene/solr/trunk/src/test/org/apache/solr/highlight/HighlighterTest.java
 Sun Sep 27 13:58:30 2009
 @@ -585,6 +585,7 @@
  args.put(hl.fl, t_text);
  args.put(hl.fragsize, 40);
  args.put(hl.snippets, 10);
 +args.put(hl.usePhraseHighlighter, false);
  
  TestHarness.LocalRequestFactory sumLRF = h.getRequestFactory(
standard, 0, 200, args);



   
   


   


-- 
- Mark

http://www.lucidimagination.com

Re: svn commit: r818618 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/core/SolrCore.java src/java/org/apache/solr/search/SolrFieldCacheMBean.java

[Chris Hostetter]

: On Sep 24, 2009, at 4:35 PM, hoss...@apache.org wrote:
:  +stats.add(instanity_count, insanity.length);
: 
: hoss spelling alert!  insanity!

no ... insanity would be if i actually managed to spell insanity correctly 
every time.

thanks for fixing in Shalin.


-Hoss

Re: svn commit: r818618 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/core/SolrCore.java src/java/org/apache/solr/search/SolrFieldCacheMBean.java

[Erik Hatcher]

On Sep 24, 2009, at 4:35 PM, hoss...@apache.org wrote:

+stats.add(instanity_count, insanity.length);


hoss spelling alert!  insanity!

Erik

Re: svn commit: r808988 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/request/PHPSerializedResponseWriter.java

[Yonik Seeley]

On Fri, Sep 11, 2009 at 8:37 PM, Chris Hostetter
hossman_luc...@fucit.org wrote:
 Isn't that an argument in favor of having an explicit option to control
 how we do the counting? otherwise we're still at risk of the scenerio i
 discribed (ie: jetty fixes the byte conversion code, but we're still
 counting the bytes wrong for them)

I tried testing w/ Jetty7 RC5 but didn't get too far (couldn't bring it up).

I've attached an additional patch to SOLR-1091 that adds a boolean
CESU8 system property.

-Yonik
http://www.lucidimagination.com

Re: svn commit: r808988 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/request/PHPSerializedResponseWriter.java

[Chris Hostetter]

: A code point (unicode character) outside of the BMP (basic
: multilingual plane, fits in 16 bits) is represented as two java chars
: - a surrogate pair.  It's a single logical character - see
: String.codePointAt().  In correct UTF-8 it should be encoded as a
: single code point... but Jetty is ignoring the fact that it's a
: surrogate pair and encoding each Java char as it's own code point...
: this is often called modified-UTF8 or CESU-8.
: 
: So... say you have this incorrect CESU-8 that is masquerading as
: UTF-8: all is not lost.
...

I must be missunderstanding something still ... based on your description, 
it sounds like it shouldn't matter if the encoder knows that it's one 
logical character or not, either way it should wind up outputing the same 
number of bytes

Except that if that were the case, why would we have had this bug in the 
first place?  clearly i'm still missunderstanding something.

: Bottom line - if we correctly encapsulate whatever the servlet
: container is writing, it's certainly possible for clients to use the
: output correctly.

I still come back to not liking that this is a hardcoded hack just for 
jetty ... it seems like it would be easy for a future version of jetty to 
change this behavior in some way that makes solr start breaking -- jetty 
could fix this bug and break solr's byte counting ... that doesn't seem 
cool

why don't we just output the raw bytes ourselves?  the code to generate 
the byte[] was/is allready there, we're just ignoring it and only using 
the length.



-Hoss

Re: svn commit: r808988 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/request/PHPSerializedResponseWriter.java

[Robert Muir]

On Fri, Sep 11, 2009 at 5:06 PM, Chris Hostetter
hossman_luc...@fucit.org wrote:

 I must be missunderstanding something still ... based on your description,
 it sounds like it shouldn't matter if the encoder knows that it's one
 logical character or not, either way it should wind up outputing the same
 number of bytes

yes, the # of bytes is different: 6 bytes versus 4 bytes in UTF-8

-- 
Robert Muir
rcm...@gmail.com

Re: svn commit: r808988 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/request/PHPSerializedResponseWriter.java

[Yonik Seeley]

On Fri, Sep 11, 2009 at 5:06 PM, Chris Hostetter
hossman_luc...@fucit.org wrote:
 why don't we just output the raw bytes ourselves?

That would require writing TextResponseWriter and friends as binary
writers, right?  Or did you have a different way in mind for injecting
bytes into the output stream?

-Yonik
http://www.lucidimagination.com

Re: svn commit: r808988 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/request/PHPSerializedResponseWriter.java

[Chris Hostetter]

:  why don't we just output the raw bytes ourselves?
: 
: That would require writing TextResponseWriter and friends as binary
: writers, right?  Or did you have a different way in mind for injecting
: bytes into the output stream?


Grr you're right. i got so turned arround thinking about 
counting the bytes and encapsulating it all in the PHPS markup i forgot 
that we still want/need the *raw* bytes output as part of the character 
stream ... not some sort of string representation of the bytes ... wow 
that sounds even more comfusing when i look at it on paper.

character data sucks.

I still repeat my earlier vote: let's yank this patch and just document 
that byte counts for strings included in the PHPS format are only accurate 
for characters outside the BMP if the servlet container being used writes 
them correctly -- that seems like a totally fair requirement to having in 
place, and points the figner squarly where in belongs (without putting us 
at risk for having broken code if/when jetty fixes this problem.

Alternately: have an option and/or system property to force/disable this 
behavior even if jetty.home isn't/is set.

Even if we change nothing else: there needs to be a big fat freaking 
disclaimer in the javadocs for the writer that it's looking at the 
jetty.home variable, and explaining why.



-Hoss

Re: svn commit: r808988 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/request/PHPSerializedResponseWriter.java

[Yonik Seeley]

On Fri, Sep 11, 2009 at 6:21 PM, Donovan Jimenez
djime...@conduit-it.com wrote:
 Is it possible (and would it even help)  to normalize all strings with
 regards to surrogate pairs at indexing time instead?

Already done, in a way... there's only one way to represent a
character outside the BMP in UTF-16 (which is the in-memory encoding
used by Java String).  Unless I misunderstood what you meant by
normalization.

-Yonik
http://www.lucidimagination.com

Re: svn commit: r808988 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/request/PHPSerializedResponseWriter.java

[Yonik Seeley]

On Tue, Sep 8, 2009 at 7:46 PM, Chris Hostetter
hossman_luc...@fucit.org wrote:
 if the container can't correctly output
 some characters, i see no reason to hide the bug

Another problem is that it won't reliably break.  The bug breaks our
encapsulation (before the patch) and thus the client reads the wrong
number of chars for the string, and who knows what happens after that.
 The majority of the time will result in an exception, but it really
depends.  This is the type of stuff (buffer underflows / overflows)
that could be used to mess with security too... a carefully crafted
request could inject / change fields in the response and have it look
valid.

-Yonik
http://www.lucidimagination.com

Re: svn commit: r808988 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/request/PHPSerializedResponseWriter.java

[Donovan Jimenez]

you are correct, it was my misunderstanding of the problem - now that  
I've read more than I ever wanted to know about UCS-2, UTF-16 and  
modified UTF-8, I'm more upto speed.


Thanks for the patience.

On Sep 11, 2009, at 6:32 PM, Yonik Seeley wrote:


On Fri, Sep 11, 2009 at 6:21 PM, Donovan Jimenez
djime...@conduit-it.com wrote:
Is it possible (and would it even help)  to normalize all strings  
with

regards to surrogate pairs at indexing time instead?


Already done, in a way... there's only one way to represent a
character outside the BMP in UTF-16 (which is the in-memory encoding
used by Java String).  Unless I misunderstood what you meant by
normalization.

-Yonik
http://www.lucidimagination.com

Re: svn commit: r808988 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/request/PHPSerializedResponseWriter.java

[Chris Hostetter]

:  if the container can't correctly output
:  some characters, i see no reason to hide the bug
: 
: Another problem is that it won't reliably break.  The bug breaks our
: encapsulation (before the patch) and thus the client reads the wrong
: number of chars for the string, and who knows what happens after that.
:  The majority of the time will result in an exception, but it really
: depends.  This is the type of stuff (buffer underflows / overflows)
: that could be used to mess with security too... a carefully crafted
: request could inject / change fields in the response and have it look
: valid.

Isn't that an argument in favor of having an explicit option to control 
how we do the counting? otherwise we're still at risk of the scenerio i 
discribed (ie: jetty fixes the byte conversion code, but we're still 
counting the bytes wrong for them)

with an explicit option we put control in the hands of the solr admin to 
decide what's right for them (we can even give them a little shell script 
to test it with)


-Hoss

Re: svn commit: r808988 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/request/PHPSerializedResponseWriter.java

[Chris Hostetter]

:  : +  static boolean modifiedUTF8 = System.getProperty(jetty.home) != null;
: 
:  ...that seems really hackish to me, particularly since for all we know
:  there are other servlet containers that might have the same problem.
: 
: Yeah, it is.
: But it's not really a valid option, it's a bug/limitation in the
: servlet container IMO.  It would also suck to bloat configuration (and
: users brains) with options that don't really control anything, except
: that they must correctly match it up with how their servlet container
: behaves.  And this doesn't actually fix everything - it simply makes
: it such that encapsulation at the transport layer isn't broken - the
: end user will still be getting back incorrect UTF8.

my suggestion for adding an explicit option for the modifiedUTF8 behavior 
to the phps response writer was on the (miss)understanding that the value 
was always correct, it was just the length calculation that was being done 
wrong by jetty (and correct by other implementations) so if we noticed we 
were running in jetty we'd replace the length calculation with our own 
(which might be less efficient).

but i guess i don't really understand the patch ... actually, the more i 
look at it the less i understand it

The modifiedUTF8 boolean only influence the numeric length returned as the 
s option ... the actaully val string is still written as is by the 
servlet container.  you've also changed the behavior so that even when 
false==modifiedUTF8, the length is now computed differently then before 
the patch (using UnicodeUtil.UTF16toUTF8) ... it's this second change i 
don't understand based on the context of the bug: why does the length 
value need to be computed differnetly for non-jetty implemntations?

: containers hands and do it all ourselves.  Or just don't support any
: servlet containers that can't handle code points outside the BMP?  Or

I'm in favor of this approach .. if the container can't correctly output 
some characters, i see no reason to hide the bug -- if somebody else wants 
to code arround the bug by doing it all in solr that's fine, but untill 
then i don't see an advantage in outputing the correct number of bytes for 
a garbage string -- anybody who really cares is going to need to switch to 
a differnet servlet container anyway.

: is there simply a Jetty config option we've been missing.  It's hard
: to believe that such a popular servlet container can't handle this.

hey, i don't even understand the bug enough to know what to search for to 
try and find an option.



-Hoss

Re: svn commit: r808988 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/request/PHPSerializedResponseWriter.java

[Yonik Seeley]

On Tue, Sep 8, 2009 at 7:46 PM, Chris Hostetterhossman_luc...@fucit.org wrote:
 The modifiedUTF8 boolean only influence the numeric length returned as the
 s option ... the actaully val string is still written as is by the
 servlet container.

Yep.

A code point (unicode character) outside of the BMP (basic
multilingual plane, fits in 16 bits) is represented as two java chars
- a surrogate pair.  It's a single logical character - see
String.codePointAt().  In correct UTF-8 it should be encoded as a
single code point... but Jetty is ignoring the fact that it's a
surrogate pair and encoding each Java char as it's own code point...
this is often called modified-UTF8 or CESU-8.

So... say you have this incorrect CESU-8 that is masquerading as
UTF-8: all is not lost.
- A decoder can unambiguously recognize that the characters decoded
actually form a surrogate pair and correctly decode - but I don't know
if there are aby requirements to do so (doubt it), and I don't know
which do so.
- A decoder decoding into UTF-16 (like Java Strings) will correctly
decode anyway, even if treating each code point in the pair as
separate.

PHP5 doesn't even do unicode, so it won't care.
PHP6 apparently has unicode support - don't know much about it.

Bottom line - if we correctly encapsulate whatever the servlet
container is writing, it's certainly possible for clients to use the
output correctly.

 you've also changed the behavior so that even when
 false==modifiedUTF8, the length is now computed differently then before
 the patch (using UnicodeUtil.UTF16toUTF8) ... it's this second change i
 don't understand based on the context of the bug: why does the length
 value need to be computed differnetly for non-jetty implemntations?

It will be the same length for non-jetty implementations - I just
rewrote the entire method and used the Lucene UTF16toUTF8 for
performance reasons.  (bad developer, bad!)

-Yonik
http://www.lucidimagination.com

Re: svn commit: r808988 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/request/PHPSerializedResponseWriter.java

[Yonik Seeley]

On Thu, Sep 3, 2009 at 8:24 PM, Chris Hostetterhossman_luc...@fucit.org wrote:

 : +61. SOLR-1091: Jetty's use of CESU-8 for code points outside the BMP
 : +    resulted in invalid output from the serialized PHP writer. (yonik)

        ...

 : +  static boolean modifiedUTF8 = System.getProperty(jetty.home) != null;

 ...that seems really hackish to me, particularly since for all we know
 there are other servlet containers that might have the same problem.

Yeah, it is.
But it's not really a valid option, it's a bug/limitation in the
servlet container IMO.  It would also suck to bloat configuration (and
users brains) with options that don't really control anything, except
that they must correctly match it up with how their servlet container
behaves.  And this doesn't actually fix everything - it simply makes
it such that encapsulation at the transport layer isn't broken - the
end user will still be getting back incorrect UTF8.

I guess one better fix is to take the UTF8 encoding out of the servlet
containers hands and do it all ourselves.  Or just don't support any
servlet containers that can't handle code points outside the BMP?  Or
is there simply a Jetty config option we've been missing.  It's hard
to believe that such a popular servlet container can't handle this.

-Yonik
http://www.lucidimagination.com

Re: svn commit: r808988 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/request/PHPSerializedResponseWriter.java

[Chris Hostetter]

: +61. SOLR-1091: Jetty's use of CESU-8 for code points outside the BMP
: +resulted in invalid output from the serialized PHP writer. (yonik)

...

: +  static boolean modifiedUTF8 = System.getProperty(jetty.home) != null;

...that seems really hackish to me, particularly since for all we know 
there are other servlet containers that might have the same problem.

Wouldn't it make more sense to add an init param for the 
PHPSerializedResponseWriter that lets people set this deterministicly in 
solrconfig.xml?  (and then we could set it and comment on it in the 
example configs so things are more transparent)

-Hoss

Re: svn commit: r787205 - in /lucene/solr/trunk/contrib/dataimporthandler: CHANGES.txt src/main/java/org/apache/solr/handler/dataimport/DataImportHandler.java src/main/java/org/apache/solr/handler/dat

[Erik Hatcher]

On Jun 22, 2009, at 7:27 AM, no...@apache.org wrote:

Author: noble
Date: Mon Jun 22 11:27:53 2009
New Revision: 787205

URL: http://svn.apache.org/viewvc?rev=787205view=rev
Log:
36.SOLR-1234: Multiple DIH does not work because all of them write  
to dataimport.properties. Use the handler name as the properties  
file name


Are there any issues with folks upgrading and delta queries having  
issues because it's looking for a different .properties file?



+  if(name.startsWith(/)){
+myName = name.substring(1);
+  }


Note that request handler mappings can be with more than just a  
leading slash.  This needs to be adjusted to replace all slashes with  
underscore or something like that.  For example, in our latest project  
we're using /indexer/articles as the DIH mapping for one source.


An alternative is to use a single dataimport.properties file and look  
for keys based on the request handler mapping name.


I'm going to re-open this issue to let the multiple-slash issue get  
resolved.


Erik

Re: svn commit: r787205 - in /lucene/solr/trunk/contrib/dataimporthandler: CHANGES.txt src/main/java/org/apache/solr/handler/dataimport/DataImportHandler.java src/main/java/org/apache/solr/handler/d

[Noble Paul നോബിള്‍ नोब्ळ्]

using single properties file is error prone. They are not synchronized
and writes may happen at anytime

On Tue, Jun 23, 2009 at 6:55 AM, Erik Hatchere...@ehatchersolutions.com wrote:

 On Jun 22, 2009, at 7:27 AM, no...@apache.org wrote:

 Author: noble
 Date: Mon Jun 22 11:27:53 2009
 New Revision: 787205

 URL: http://svn.apache.org/viewvc?rev=787205view=rev
 Log:
 36.SOLR-1234: Multiple DIH does not work because all of them write to
 dataimport.properties. Use the handler name as the properties file name

 Are there any issues with folks upgrading and delta queries having issues
 because it's looking for a different .properties file?

 +          if(name.startsWith(/)){
 +            myName = name.substring(1);
 +          }

 Note that request handler mappings can be with more than just a leading
 slash.  This needs to be adjusted to replace all slashes with underscore or
 something like that.  For example, in our latest project we're using
 /indexer/articles as the DIH mapping for one source.

 An alternative is to use a single dataimport.properties file and look for
 keys based on the request handler mapping name.

 I'm going to re-open this issue to let the multiple-slash issue get
 resolved.

        Erik





-- 
-
Noble Paul | Principal Engineer| AOL | http://aol.com

Re: svn commit: r779144 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/handler/component/TermVectorComponent.java src/test/org/apache/solr/handler/component/TermVectorComponentTest.java

[Erik Hatcher]

Grant - I don't think you meant to remove Yonik's addition to  
CHANGES.txt...


On May 27, 2009, at 8:34 AM, gsing...@apache.org wrote:

Author: gsingers
Date: Wed May 27 12:34:09 2009
New Revision: 779144

URL: http://svn.apache.org/viewvc?rev=779144view=rev
Log:
SOLR-1188: Minor efficiency improvement in TermVectorComponent

Modified:
   lucene/solr/trunk/CHANGES.txt
   lucene/solr/trunk/src/java/org/apache/solr/handler/component/ 
TermVectorComponent.java
   lucene/solr/trunk/src/test/org/apache/solr/handler/component/ 
TermVectorComponentTest.java


Modified: lucene/solr/trunk/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/lucene/solr/trunk/CHANGES.txt?rev=779144r1=779143r2=779144view=diff
= 
= 
= 
= 
= 
= 
= 
= 
==

--- lucene/solr/trunk/CHANGES.txt (original)
+++ lucene/solr/trunk/CHANGES.txt Wed May 27 12:34:09 2009
@@ -269,10 +269,6 @@
12. SOLR-1165: Use Lucene Filters and pass them down to the Lucene
search methods to filter earlier and improve performance. (yonik)

-13. SOLR-: Use per-segment sorting to share fieldcache elements
-across unchanged segments.  This saves memory and reduces
-commit times for incremental updates to the index. (yonik)
-

Re: svn commit: r779144 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/handler/component/TermVectorComponent.java src/test/org/apache/solr/handler/component/TermVectorComponentTest.java

[Grant Ingersoll]

Nope, I'll fix.  I thought it merged.

On May 27, 2009, at 8:39 AM, Erik Hatcher wrote:

Grant - I don't think you meant to remove Yonik's addition to  
CHANGES.txt...


On May 27, 2009, at 8:34 AM, gsing...@apache.org wrote:

Author: gsingers
Date: Wed May 27 12:34:09 2009
New Revision: 779144

URL: http://svn.apache.org/viewvc?rev=779144view=rev
Log:
SOLR-1188: Minor efficiency improvement in TermVectorComponent

Modified:
  lucene/solr/trunk/CHANGES.txt
  lucene/solr/trunk/src/java/org/apache/solr/handler/component/ 
TermVectorComponent.java
  lucene/solr/trunk/src/test/org/apache/solr/handler/component/ 
TermVectorComponentTest.java


Modified: lucene/solr/trunk/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/lucene/solr/trunk/CHANGES.txt?rev=779144r1=779143r2=779144view=diff
= 
= 
= 
= 
= 
= 
= 
= 
= 
=

--- lucene/solr/trunk/CHANGES.txt (original)
+++ lucene/solr/trunk/CHANGES.txt Wed May 27 12:34:09 2009
@@ -269,10 +269,6 @@
12. SOLR-1165: Use Lucene Filters and pass them down to the Lucene
   search methods to filter earlier and improve performance. (yonik)

-13. SOLR-: Use per-segment sorting to share fieldcache elements
-across unchanged segments.  This saves memory and reduces
-commit times for incremental updates to the index. (yonik)
-




--
Grant Ingersoll
http://www.lucidimagination.com/

Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)  
using Solr/Lucene:

http://www.lucidimagination.com/search

Re: svn commit: r742106 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/handler/XMLLoader.java

[Koji Sekiguchi]

Chris Hostetter wrote:

: I think it is true. Unfortunately I didn't have time to write test for it, but
: I
: tested it by manual when I committed it... Do you see it doesn't work?

Hmmm. 


It does work ... I don't understand why it works, but it works.


-Hoss


  


It should work. I don't understand why you think so...

Koji

Re: svn commit: r742106 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/handler/XMLLoader.java

[Chris Hostetter]

:  
: +29. SOLR-976: deleteByQuery is ignored when deleteById is placed prior to 
deleteByQuery in a delete.
: +Now both delete by id and delete by query can be specified at the same 
time as follows. (koji)
: +  delete
: +id05991/idid06000/id
: +queryoffice:Bridgewater/queryqueryoffice:Osaka/query
: +  /delete
: +

is this really true?  the patch only modifies XMLLoader, but don't 
DeleteUpdateCommand and RunUpdateProcessor also need to be changed to make 
this work ... there's was no test commited with this patch so i'm not 
convinced, and i see this code in RunUpdateProcessor...

if( cmd.id != null ) {
  updateHandler.delete(cmd);
}
else {
  updateHandler.deleteByQuery(cmd);
}



-Hoss

Re: svn commit: r742106 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/handler/XMLLoader.java

[Koji Sekiguchi]

 is this really true?

I think it is true. Unfortunately I didn't have time to write test for 
it, but I

tested it by manual when I committed it... Do you see it doesn't work?

Koji

Chris Hostetter wrote:
:  
: +29. SOLR-976: deleteByQuery is ignored when deleteById is placed prior to deleteByQuery in a delete.

: +Now both delete by id and delete by query can be specified at the same 
time as follows. (koji)
: +  delete
: +id05991/idid06000/id
: +queryoffice:Bridgewater/queryqueryoffice:Osaka/query
: +  /delete
: +

is this really true?  the patch only modifies XMLLoader, but don't 
DeleteUpdateCommand and RunUpdateProcessor also need to be changed to make 
this work ... there's was no test commited with this patch so i'm not 
convinced, and i see this code in RunUpdateProcessor...


if( cmd.id != null ) {
  updateHandler.delete(cmd);
}
else {
  updateHandler.deleteByQuery(cmd);
}



-Hoss


  

Re: svn commit: r741097 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/handler/component/QueryComponent.java

[Ryan McKinley]

I'm not sure this is right

It is the query parser that knows if 'q' is required or not.


On Feb 5, 2009, at 8:10 AM, sha...@apache.org wrote:


Author: shalin
Date: Thu Feb  5 13:10:33 2009
New Revision: 741097

URL: http://svn.apache.org/viewvc?rev=741097view=rev
Log:
Reverting SOLR-435 (always run tests before committing)

Modified:
   lucene/solr/trunk/CHANGES.txt
   lucene/solr/trunk/src/java/org/apache/solr/handler/component/ 
QueryComponent.java


Modified: lucene/solr/trunk/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/lucene/solr/trunk/CHANGES.txt?rev=741097r1=741096r2=741097view=diff
= 
= 
= 
= 
= 
= 
= 
= 
==

--- lucene/solr/trunk/CHANGES.txt (original)
+++ lucene/solr/trunk/CHANGES.txt Thu Feb  5 13:10:33 2009
@@ -236,8 +236,6 @@
27. SOLR-991: Better error message when parsing solrconfig.xml fails  
due to malformed XML. Error message notes the name

of the file being parsed. (Michael Henson via shalin)

-28. SOLR-435: NullPointerException with no query or empty query  
(ryan, Lars Kotthoff via shalin)

-

Other Changes
--

Modified: lucene/solr/trunk/src/java/org/apache/solr/handler/ 
component/QueryComponent.java

URL: 
http://svn.apache.org/viewvc/lucene/solr/trunk/src/java/org/apache/solr/handler/component/QueryComponent.java?rev=741097r1=741096r2=741097view=diff
= 
= 
= 
= 
= 
= 
= 
= 
==
--- lucene/solr/trunk/src/java/org/apache/solr/handler/component/ 
QueryComponent.java (original)
+++ lucene/solr/trunk/src/java/org/apache/solr/handler/component/ 
QueryComponent.java Thu Feb  5 13:10:33 2009

@@ -74,12 +74,7 @@
defType = defType==null ? OldLuceneQParserPlugin.NAME : defType;

if (rb.getQueryString() == null) {
-  rb.setQueryString(params.get(CommonParams.Q));
-  String queryString = params.get(CommonParams.Q);
-  if (queryString == null) {
-throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
-Missing or empty required parameter: q);
-  }
+  rb.setQueryString( params.get( CommonParams.Q ) );
}

try {

Re: svn commit: r741097 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/handler/component/QueryComponent.java

[Shalin Shekhar Mangar]

Yes, sorry about that. I committed this first and ran the tests later. I've
reverted this change for now and I'll re-open the issue.

On Thu, Feb 5, 2009 at 7:50 PM, Ryan McKinley ryan...@gmail.com wrote:

 I'm not sure this is right

 It is the query parser that knows if 'q' is required or not.



 On Feb 5, 2009, at 8:10 AM, sha...@apache.org wrote:

  Author: shalin
 Date: Thu Feb  5 13:10:33 2009
 New Revision: 741097

 URL: http://svn.apache.org/viewvc?rev=741097view=rev
 Log:
 Reverting SOLR-435 (always run tests before committing)

 Modified:
   lucene/solr/trunk/CHANGES.txt

 lucene/solr/trunk/src/java/org/apache/solr/handler/component/QueryComponent.java

 Modified: lucene/solr/trunk/CHANGES.txt
 URL:
 http://svn.apache.org/viewvc/lucene/solr/trunk/CHANGES.txt?rev=741097r1=741096r2=741097view=diff

 ==
 --- lucene/solr/trunk/CHANGES.txt (original)
 +++ lucene/solr/trunk/CHANGES.txt Thu Feb  5 13:10:33 2009
 @@ -236,8 +236,6 @@
 27. SOLR-991: Better error message when parsing solrconfig.xml fails due
 to malformed XML. Error message notes the name
of the file being parsed. (Michael Henson via shalin)

 -28. SOLR-435: NullPointerException with no query or empty query (ryan,
 Lars Kotthoff via shalin)
 -

 Other Changes
 --

 Modified:
 lucene/solr/trunk/src/java/org/apache/solr/handler/component/QueryComponent.java
 URL:
 http://svn.apache.org/viewvc/lucene/solr/trunk/src/java/org/apache/solr/handler/component/QueryComponent.java?rev=741097r1=741096r2=741097view=diff

 ==
 ---
 lucene/solr/trunk/src/java/org/apache/solr/handler/component/QueryComponent.java
 (original)
 +++
 lucene/solr/trunk/src/java/org/apache/solr/handler/component/QueryComponent.java
 Thu Feb  5 13:10:33 2009
 @@ -74,12 +74,7 @@
defType = defType==null ? OldLuceneQParserPlugin.NAME : defType;

if (rb.getQueryString() == null) {
 -  rb.setQueryString(params.get(CommonParams.Q));
 -  String queryString = params.get(CommonParams.Q);
 -  if (queryString == null) {
 -throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
 -Missing or empty required parameter: q);
 -  }
 +  rb.setQueryString( params.get( CommonParams.Q ) );
}

try {






-- 
Regards,
Shalin Shekhar Mangar.

Re: svn commit: r702314 - in /lucene/solr/trunk/client/java/solrj: CHANGES.txt src/org/apache/solr/client/solrj/util/ClientUtils.java

[Ryan McKinley]

how about keeping one function, but adding:   Character.whiteSpace( c )?

Do you need the function to also escape whitespace?


On Oct 6, 2008, at 8:36 PM, [EMAIL PROTECTED] wrote:


Author: koji
Date: Mon Oct  6 17:36:49 2008
New Revision: 702314

URL: http://svn.apache.org/viewvc?rev=702314view=rev
Log:
SOLR-794: added escape() method to ClientUtils.

Modified:
   lucene/solr/trunk/client/java/solrj/CHANGES.txt
   lucene/solr/trunk/client/java/solrj/src/org/apache/solr/client/ 
solrj/util/ClientUtils.java


Modified: lucene/solr/trunk/client/java/solrj/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/lucene/solr/trunk/client/java/solrj/CHANGES.txt?rev=702314r1=702313r2=702314view=diff
= 
= 
= 
= 
= 
= 
= 
= 
==

--- lucene/solr/trunk/client/java/solrj/CHANGES.txt (original)
+++ lucene/solr/trunk/client/java/solrj/CHANGES.txt Mon Oct  6  
17:36:49 2008

@@ -18,6 +18,8 @@
New Features
--

+ 1. SOLR-794: added escape() method to ClientUtils. (koji)
+
Optimizations
--


Modified: lucene/solr/trunk/client/java/solrj/src/org/apache/solr/ 
client/solrj/util/ClientUtils.java

URL: 
http://svn.apache.org/viewvc/lucene/solr/trunk/client/java/solrj/src/org/apache/solr/client/solrj/util/ClientUtils.java?rev=702314r1=702313r2=702314view=diff
= 
= 
= 
= 
= 
= 
= 
= 
==
--- lucene/solr/trunk/client/java/solrj/src/org/apache/solr/client/ 
solrj/util/ClientUtils.java (original)
+++ lucene/solr/trunk/client/java/solrj/src/org/apache/solr/client/ 
solrj/util/ClientUtils.java Mon Oct  6 17:36:49 2008

@@ -191,15 +191,32 @@
  private static final Pattern escapePattern = Pattern.compile( (\ 
\W) );


  /**
-   * See: http://lucene.apache.org/java/docs/queryparsersyntax.html#Escaping 
 Special Characters

+   * Non-word characters are escaped by a preceding code\/code.
   */
  public static String escapeQueryChars( String input )
  {
Matcher matcher = escapePattern.matcher( input );
return matcher.replaceAll( $1 );
  }
-

+  /**
+   * See: http://lucene.apache.org/java/docs/queryparsersyntax.html#Escaping 
 Special Characters

+   */
+  public static String escape(String s) {
+StringBuffer sb = new StringBuffer();
+for (int i = 0; i  s.length(); i++) {
+  char c = s.charAt(i);
+  // These characters are part of the query syntax and must be  
escaped
+  if (c == '\\' || c == '+' || c == '-' || c == '!' || c == '('  
|| c == ')' || c == ':'
+|| c == '^' || c == '[' || c == ']' || c == '\' || c ==  
'{' || c == '}' || c == '~'

+|| c == '*' || c == '?' || c == '|' || c == '') {
+sb.append('\\');
+  }
+  sb.append(c);
+}
+return sb.toString();
+  }
+
  public static String toQueryString( SolrParams params, boolean  
xml ) {

StringBuilder sb = new StringBuilder(128);
try {

Re: svn commit: r702314 - in /lucene/solr/trunk/client/java/solrj: CHANGES.txt src/org/apache/solr/client/solrj/util/ClientUtils.java

[Koji Sekiguchi]

I think keeping one function is good. Why I added the new function is 
because

I just didn't want to change runtime behavior of existing public method...

Ryan McKinley wrote:

how about keeping one function, but adding:   Character.whiteSpace( c )?

Do you need the function to also escape whitespace?


On Oct 6, 2008, at 8:36 PM, [EMAIL PROTECTED] wrote:


Author: koji
Date: Mon Oct  6 17:36:49 2008
New Revision: 702314

URL: http://svn.apache.org/viewvc?rev=702314view=rev
Log:
SOLR-794: added escape() method to ClientUtils.

Modified:
   lucene/solr/trunk/client/java/solrj/CHANGES.txt
   
lucene/solr/trunk/client/java/solrj/src/org/apache/solr/client/solrj/util/ClientUtils.java 



Modified: lucene/solr/trunk/client/java/solrj/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/lucene/solr/trunk/client/java/solrj/CHANGES.txt?rev=702314r1=702313r2=702314view=diff 

== 


--- lucene/solr/trunk/client/java/solrj/CHANGES.txt (original)
+++ lucene/solr/trunk/client/java/solrj/CHANGES.txt Mon Oct  6 
17:36:49 2008

@@ -18,6 +18,8 @@
New Features
--

+ 1. SOLR-794: added escape() method to ClientUtils. (koji)
+
Optimizations
--


Modified: 
lucene/solr/trunk/client/java/solrj/src/org/apache/solr/client/solrj/util/ClientUtils.java 

URL: 
http://svn.apache.org/viewvc/lucene/solr/trunk/client/java/solrj/src/org/apache/solr/client/solrj/util/ClientUtils.java?rev=702314r1=702313r2=702314view=diff 

== 

--- 
lucene/solr/trunk/client/java/solrj/src/org/apache/solr/client/solrj/util/ClientUtils.java 
(original)
+++ 
lucene/solr/trunk/client/java/solrj/src/org/apache/solr/client/solrj/util/ClientUtils.java 
Mon Oct  6 17:36:49 2008

@@ -191,15 +191,32 @@
  private static final Pattern escapePattern = Pattern.compile( 
(\\W) );


  /**
-   * See: 
http://lucene.apache.org/java/docs/queryparsersyntax.html#Escaping Special 
Characters

+   * Non-word characters are escaped by a preceding code\/code.
   */
  public static String escapeQueryChars( String input )
  {
Matcher matcher = escapePattern.matcher( input );
return matcher.replaceAll( $1 );
  }
-

+  /**
+   * See: 
http://lucene.apache.org/java/docs/queryparsersyntax.html#Escaping Special 
Characters

+   */
+  public static String escape(String s) {
+StringBuffer sb = new StringBuffer();
+for (int i = 0; i  s.length(); i++) {
+  char c = s.charAt(i);
+  // These characters are part of the query syntax and must be 
escaped
+  if (c == '\\' || c == '+' || c == '-' || c == '!' || c == '(' 
|| c == ')' || c == ':'
+|| c == '^' || c == '[' || c == ']' || c == '\' || c == '{' 
|| c == '}' || c == '~'

+|| c == '*' || c == '?' || c == '|' || c == '') {
+sb.append('\\');
+  }
+  sb.append(c);
+}
+return sb.toString();
+  }
+
  public static String toQueryString( SolrParams params, boolean xml ) {
StringBuilder sb = new StringBuilder(128);
try {

Re: svn commit: r702314 - in /lucene/solr/trunk/client/java/solrj: CHANGES.txt src/org/apache/solr/client/solrj/util/ClientUtils.java

[Ryan McKinley]

I agree one is better...  as long as the escape functionality is the  
same, i don't think it matters - adding the Character.isWhitespace()  
should make that true.


i say we change it and add a note in CHANGES.txt -- if there are any  
problems (i can't imagine functional ones) then we can change back.



On Oct 6, 2008, at 11:19 PM, Koji Sekiguchi wrote:

I think keeping one function is good. Why I added the new function  
is because
I just didn't want to change runtime behavior of existing public  
method...


Ryan McKinley wrote:
how about keeping one function, but adding:
Character.whiteSpace( c )?


Do you need the function to also escape whitespace?


On Oct 6, 2008, at 8:36 PM, [EMAIL PROTECTED] wrote:


Author: koji
Date: Mon Oct  6 17:36:49 2008
New Revision: 702314

URL: http://svn.apache.org/viewvc?rev=702314view=rev
Log:
SOLR-794: added escape() method to ClientUtils.

Modified:
  lucene/solr/trunk/client/java/solrj/CHANGES.txt
  lucene/solr/trunk/client/java/solrj/src/org/apache/solr/client/ 
solrj/util/ClientUtils.java


Modified: lucene/solr/trunk/client/java/solrj/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/lucene/solr/trunk/client/java/solrj/CHANGES.txt?rev=702314r1=702313r2=702314view=diff
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 


--- lucene/solr/trunk/client/java/solrj/CHANGES.txt (original)
+++ lucene/solr/trunk/client/java/solrj/CHANGES.txt Mon Oct  6  
17:36:49 2008

@@ -18,6 +18,8 @@
New Features
--

+ 1. SOLR-794: added escape() method to ClientUtils. (koji)
+
Optimizations
--


Modified: lucene/solr/trunk/client/java/solrj/src/org/apache/solr/ 
client/solrj/util/ClientUtils.java

URL: 
http://svn.apache.org/viewvc/lucene/solr/trunk/client/java/solrj/src/org/apache/solr/client/solrj/util/ClientUtils.java?rev=702314r1=702313r2=702314view=diff
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 

--- lucene/solr/trunk/client/java/solrj/src/org/apache/solr/client/ 
solrj/util/ClientUtils.java (original)
+++ lucene/solr/trunk/client/java/solrj/src/org/apache/solr/client/ 
solrj/util/ClientUtils.java Mon Oct  6 17:36:49 2008

@@ -191,15 +191,32 @@
 private static final Pattern escapePattern = Pattern.compile( (\ 
\W) );


 /**
-   * See: http://lucene.apache.org/java/docs/queryparsersyntax.html#Escaping 
 Special Characters

+   * Non-word characters are escaped by a preceding code\/code.
  */
 public static String escapeQueryChars( String input )
 {
   Matcher matcher = escapePattern.matcher( input );
   return matcher.replaceAll( $1 );
 }
-

+  /**
+   * See: http://lucene.apache.org/java/docs/queryparsersyntax.html#Escaping 
 Special Characters

+   */
+  public static String escape(String s) {
+StringBuffer sb = new StringBuffer();
+for (int i = 0; i  s.length(); i++) {
+  char c = s.charAt(i);
+  // These characters are part of the query syntax and must  
be escaped
+  if (c == '\\' || c == '+' || c == '-' || c == '!' || c ==  
'(' || c == ')' || c == ':'
+|| c == '^' || c == '[' || c == ']' || c == '\' || c ==  
'{' || c == '}' || c == '~'

+|| c == '*' || c == '?' || c == '|' || c == '') {
+sb.append('\\');
+  }
+  sb.append(c);
+}
+return sb.toString();
+  }
+
 public static String toQueryString( SolrParams params, boolean  
xml ) {

   StringBuilder sb = new StringBuilder(128);
   try {

Re: svn commit: r693498 - in /lucene/solr/trunk: CHANGES.txt build.xml lib/geronimo-stax-api_1.0_spec-1.0.1.jar lib/stax-1.2.0.jar lib/stax-api-1.0.1.jar lib/wstx-asl-3.2.7.jar

[Chris Hostetter]

For future refrence: this is a significant enough change that it would 
have been really nice to open a Jira issue first and refer to it in the 
commit message.

The Jira description could have then contained links to the mail archive 
fully explaining why the change was made (and served as a refrence point 
for linking any potential bugs that it may cause)


: Date: Tue, 09 Sep 2008 15:57:37 -
: From: [EMAIL PROTECTED]
: Reply-To: solr-dev@lucene.apache.org
: To: [EMAIL PROTECTED]
: Subject: svn commit: r693498 - in /lucene/solr/trunk: CHANGES.txt build.xml
: lib/geronimo-stax-api_1.0_spec-1.0.1.jar lib/stax-1.2.0.jar
: lib/stax-api-1.0.1.jar lib/wstx-asl-3.2.7.jar
: 
: Author: yonik
: Date: Tue Sep  9 08:57:37 2008
: New Revision: 693498
: 
: URL: http://svn.apache.org/viewvc?rev=693498view=rev
: Log:
: replace stax with geronimo API jar and woodstox impl
: 
: Added:
: lucene/solr/trunk/lib/geronimo-stax-api_1.0_spec-1.0.1.jar   (with props)
: lucene/solr/trunk/lib/wstx-asl-3.2.7.jar   (with props)
: Removed:
: lucene/solr/trunk/lib/stax-1.2.0.jar
: lucene/solr/trunk/lib/stax-api-1.0.1.jar
: Modified:
: lucene/solr/trunk/CHANGES.txt
: lucene/solr/trunk/build.xml
: 
: Modified: lucene/solr/trunk/CHANGES.txt
: URL: 
http://svn.apache.org/viewvc/lucene/solr/trunk/CHANGES.txt?rev=693498r1=693497r2=693498view=diff
: ==
: --- lucene/solr/trunk/CHANGES.txt (original)
: +++ lucene/solr/trunk/CHANGES.txt Tue Sep  9 08:57:37 2008
: @@ -658,6 +658,8 @@
:  24. Upgraded to Lucene 2.4-dev (r686801) (yonik)
:  25. Upgraded to Lucene 2.4-dev (r688745) 27-Aug-2008 (yonik)
:  26. Upgraded to Lucene 2.4-dev (r691741) 03-Sep-2008 (yonik)
: +27. Replaced the StAX reference implementation with the geronimo
: +StAX API jar, and the Woodstox StAX implementation. (yonik)
:  
:  Build
:   1. SOLR-411.  Changed the names of the Solr JARs to use the defacto 
standard JAR names based on
: 
: Modified: lucene/solr/trunk/build.xml
: URL: 
http://svn.apache.org/viewvc/lucene/solr/trunk/build.xml?rev=693498r1=693497r2=693498view=diff
: ==
: --- lucene/solr/trunk/build.xml (original)
: +++ lucene/solr/trunk/build.xml Tue Sep  9 08:57:37 2008
: @@ -305,7 +305,8 @@
:  /fileset
:  fileset dir=${lib}
:include name=commons-io-*.jar /
: -  include name=stax-*.jar /
: +  include name=*stax-*.jar /
: +  include name=wstx-*.jar /
:  /fileset
:  pathelement location=${dest}/common/
:/path
: @@ -597,7 +598,8 @@
:fileset dir=${lib}
:  include name=commons-codec-*.jar/
:  include name=commons-io-*.jar/
: -include name=stax-*.jar/
: +include name=*stax-*.jar /
: +include name=wstx-*.jar /
:/fileset
:fileset dir=${solrj-dir}/lib
:  include name=*.jar /
: 
: Added: lucene/solr/trunk/lib/geronimo-stax-api_1.0_spec-1.0.1.jar
: URL: 
http://svn.apache.org/viewvc/lucene/solr/trunk/lib/geronimo-stax-api_1.0_spec-1.0.1.jar?rev=693498view=auto
: ==
: Binary file - no diff available.
: 
: Propchange: lucene/solr/trunk/lib/geronimo-stax-api_1.0_spec-1.0.1.jar
: --
: svn:executable = *
: 
: Propchange: lucene/solr/trunk/lib/geronimo-stax-api_1.0_spec-1.0.1.jar
: --
: svn:mime-type = application/octet-stream
: 
: Added: lucene/solr/trunk/lib/wstx-asl-3.2.7.jar
: URL: 
http://svn.apache.org/viewvc/lucene/solr/trunk/lib/wstx-asl-3.2.7.jar?rev=693498view=auto
: ==
: Binary file - no diff available.
: 
: Propchange: lucene/solr/trunk/lib/wstx-asl-3.2.7.jar
: --
: svn:executable = *
: 
: Propchange: lucene/solr/trunk/lib/wstx-asl-3.2.7.jar
: --
: svn:mime-type = application/octet-stream
: 
: 



-Hoss

Re: svn commit: r689978 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/search/SolrQueryParser.java src/java/org/apache/solr/search/WildcardFilter.java src/test/org/apache/solr/Converted

[Chris Hostetter]

Are we really sure we want to do this w/o making it configurable on 
the QParser? (ala: SOLR-218)

Unless I'm missing something this change breaks back compatibility of for 
users who highlight wildcard queries.  As i recall: we even have users who 
force their prefix queries to be wildcards by using Hippo?* instead of 
Hippo* just so they can get highlighting.

+0


: Date: Thu, 28 Aug 2008 20:54:25 -
: From: [EMAIL PROTECTED]
: Reply-To: solr-dev@lucene.apache.org
: To: [EMAIL PROTECTED]
: Subject: svn commit: r689978 - in /lucene/solr/trunk: CHANGES.txt
: src/java/org/apache/solr/search/SolrQueryParser.java
: src/java/org/apache/solr/search/WildcardFilter.java
: src/test/org/apache/solr/ConvertedLegacyTest.java
: 
: Author: yonik
: Date: Thu Aug 28 13:54:24 2008
: New Revision: 689978
: 
: URL: http://svn.apache.org/viewvc?rev=689978view=rev
: Log:
: SOLR-737: use a constant score query for wildcards
: 
: Added:
: lucene/solr/trunk/src/java/org/apache/solr/search/WildcardFilter.java   
(with props)
: Modified:
: lucene/solr/trunk/CHANGES.txt
: lucene/solr/trunk/src/java/org/apache/solr/search/SolrQueryParser.java
: lucene/solr/trunk/src/test/org/apache/solr/ConvertedLegacyTest.java
: 
: Modified: lucene/solr/trunk/CHANGES.txt
: URL: 
http://svn.apache.org/viewvc/lucene/solr/trunk/CHANGES.txt?rev=689978r1=689977r2=689978view=diff
: ==
: --- lucene/solr/trunk/CHANGES.txt (original)
: +++ lucene/solr/trunk/CHANGES.txt Thu Aug 28 13:54:24 2008
: @@ -396,6 +396,10 @@
:  
:   3. SOLR-647: reference count the SolrCore uses to prevent a premature
:  close while a core is still in use.  (Henri Biestro, Noble Paul, yonik)
: +
: + 4. SOLR-737: SolrQueryParser now uses a ConstantScoreQuery for wildcard
: +queries that prevent an exception from being thrown when the number
: +of matching terms exceeds the BooleanQuery clause limit.  (yonik)
:  
:  Optimizations
:   1. SOLR-276: improve JSON writer speed. (yonik)
: 
: Modified: 
lucene/solr/trunk/src/java/org/apache/solr/search/SolrQueryParser.java
: URL: 
http://svn.apache.org/viewvc/lucene/solr/trunk/src/java/org/apache/solr/search/SolrQueryParser.java?rev=689978r1=689977r2=689978view=diff
: ==
: --- lucene/solr/trunk/src/java/org/apache/solr/search/SolrQueryParser.java 
(original)
: +++ lucene/solr/trunk/src/java/org/apache/solr/search/SolrQueryParser.java 
Thu Aug 28 13:54:24 2008
: @@ -22,6 +22,8 @@
:  import org.apache.lucene.queryParser.QueryParser;
:  import org.apache.lucene.search.ConstantScoreRangeQuery;
:  import org.apache.lucene.search.Query;
: +import org.apache.lucene.search.WildcardQuery;
: +import org.apache.lucene.search.ConstantScoreQuery;
:  import org.apache.lucene.analysis.Analyzer;
:  import org.apache.solr.common.SolrException;
:  import org.apache.solr.schema.FieldType;
: @@ -144,4 +146,12 @@
:  return new ConstantScorePrefixQuery(t);
:}
:  
: +  protected Query getWildcardQuery(String field, String termStr) throws 
ParseException {
: +Query q = super.getWildcardQuery(field, termStr);
: +if (q instanceof WildcardQuery) {
: +  // use a constant score query to avoid overflowing clauses
: +  return new ConstantScoreQuery(new 
WildcardFilter(((WildcardQuery)q).getTerm()));
: +}
: +return q;
: +  }
:  }
: 
: Added: lucene/solr/trunk/src/java/org/apache/solr/search/WildcardFilter.java
: URL: 
http://svn.apache.org/viewvc/lucene/solr/trunk/src/java/org/apache/solr/search/WildcardFilter.java?rev=689978view=auto
: ==
: --- lucene/solr/trunk/src/java/org/apache/solr/search/WildcardFilter.java 
(added)
: +++ lucene/solr/trunk/src/java/org/apache/solr/search/WildcardFilter.java Thu 
Aug 28 13:54:24 2008
: @@ -0,0 +1,103 @@
: +/**
: + * Licensed to the Apache Software Foundation (ASF) under one or more
: + * contributor license agreements.  See the NOTICE file distributed with
: + * this work for additional information regarding copyright ownership.
: + * The ASF licenses this file to You under the Apache License, Version 2.0
: + * (the License); you may not use this file except in compliance with
: + * the License.  You may obtain a copy of the License at
: + *
: + * http://www.apache.org/licenses/LICENSE-2.0
: + *
: + * Unless required by applicable law or agreed to in writing, software
: + * distributed under the License is distributed on an AS IS BASIS,
: + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
: + * See the License for the specific language governing permissions and
: + * limitations under the License.
: + */
: +
: +package org.apache.solr.search;
: +
: +import org.apache.lucene.search.Filter;
: +import org.apache.lucene.search.DocIdSet;
: +import org.apache.lucene.search.WildcardTermEnum;
: +import

Re: svn commit: r689978 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/search/SolrQueryParser.java src/java/org/apache/solr/search/WildcardFilter.java src/test/org/apache/solr/Converted

[Yonik Seeley]

On Fri, Aug 29, 2008 at 12:18 PM, Chris Hostetter
[EMAIL PROTECTED] wrote:
 Are we really sure we want to do this w/o making it configurable on
 the QParser? (ala: SOLR-218)

I thought about that, but these expanding term queries that have no
bounds are really broken... people who depend on them are playing with
fire.  Imagine everything working fine for months, docs being
occasionally added, until *boom* a magic limit is hit.

I wouldn't be opposed to a config option I guess... but I think the
default should be something that doesn't unpredictably break (so it
still wouldn't be 100% back compatible).

IMO, Highlighting really needs to be fixed to go through a different
mechanism than extractTerms().

-Yonik

 Unless I'm missing something this change breaks back compatibility of for
 users who highlight wildcard queries.  As i recall: we even have users who
 force their prefix queries to be wildcards by using Hippo?* instead of
 Hippo* just so they can get highlighting.

 +0


 : Date: Thu, 28 Aug 2008 20:54:25 -
 : From: [EMAIL PROTECTED]
 : Reply-To: solr-dev@lucene.apache.org
 : To: [EMAIL PROTECTED]
 : Subject: svn commit: r689978 - in /lucene/solr/trunk: CHANGES.txt
 : src/java/org/apache/solr/search/SolrQueryParser.java
 : src/java/org/apache/solr/search/WildcardFilter.java
 : src/test/org/apache/solr/ConvertedLegacyTest.java
 :
 : Author: yonik
 : Date: Thu Aug 28 13:54:24 2008
 : New Revision: 689978
 :
 : URL: http://svn.apache.org/viewvc?rev=689978view=rev
 : Log:
 : SOLR-737: use a constant score query for wildcards
 :
 : Added:
 : lucene/solr/trunk/src/java/org/apache/solr/search/WildcardFilter.java   
 (with props)
 : Modified:
 : lucene/solr/trunk/CHANGES.txt
 : lucene/solr/trunk/src/java/org/apache/solr/search/SolrQueryParser.java
 : lucene/solr/trunk/src/test/org/apache/solr/ConvertedLegacyTest.java
 :
 : Modified: lucene/solr/trunk/CHANGES.txt
 : URL: 
 http://svn.apache.org/viewvc/lucene/solr/trunk/CHANGES.txt?rev=689978r1=689977r2=689978view=diff
 : 
 ==
 : --- lucene/solr/trunk/CHANGES.txt (original)
 : +++ lucene/solr/trunk/CHANGES.txt Thu Aug 28 13:54:24 2008
 : @@ -396,6 +396,10 @@
 :
 :   3. SOLR-647: reference count the SolrCore uses to prevent a premature
 :  close while a core is still in use.  (Henri Biestro, Noble Paul, yonik)
 : +
 : + 4. SOLR-737: SolrQueryParser now uses a ConstantScoreQuery for wildcard
 : +queries that prevent an exception from being thrown when the number
 : +of matching terms exceeds the BooleanQuery clause limit.  (yonik)
 :
 :  Optimizations
 :   1. SOLR-276: improve JSON writer speed. (yonik)
 :
 : Modified: 
 lucene/solr/trunk/src/java/org/apache/solr/search/SolrQueryParser.java
 : URL: 
 http://svn.apache.org/viewvc/lucene/solr/trunk/src/java/org/apache/solr/search/SolrQueryParser.java?rev=689978r1=689977r2=689978view=diff
 : 
 ==
 : --- lucene/solr/trunk/src/java/org/apache/solr/search/SolrQueryParser.java 
 (original)
 : +++ lucene/solr/trunk/src/java/org/apache/solr/search/SolrQueryParser.java 
 Thu Aug 28 13:54:24 2008
 : @@ -22,6 +22,8 @@
 :  import org.apache.lucene.queryParser.QueryParser;
 :  import org.apache.lucene.search.ConstantScoreRangeQuery;
 :  import org.apache.lucene.search.Query;
 : +import org.apache.lucene.search.WildcardQuery;
 : +import org.apache.lucene.search.ConstantScoreQuery;
 :  import org.apache.lucene.analysis.Analyzer;
 :  import org.apache.solr.common.SolrException;
 :  import org.apache.solr.schema.FieldType;
 : @@ -144,4 +146,12 @@
 :  return new ConstantScorePrefixQuery(t);
 :}
 :
 : +  protected Query getWildcardQuery(String field, String termStr) throws 
 ParseException {
 : +Query q = super.getWildcardQuery(field, termStr);
 : +if (q instanceof WildcardQuery) {
 : +  // use a constant score query to avoid overflowing clauses
 : +  return new ConstantScoreQuery(new 
 WildcardFilter(((WildcardQuery)q).getTerm()));
 : +}
 : +return q;
 : +  }
 :  }
 :
 : Added: lucene/solr/trunk/src/java/org/apache/solr/search/WildcardFilter.java
 : URL: 
 http://svn.apache.org/viewvc/lucene/solr/trunk/src/java/org/apache/solr/search/WildcardFilter.java?rev=689978view=auto
 : 
 ==
 : --- lucene/solr/trunk/src/java/org/apache/solr/search/WildcardFilter.java 
 (added)
 : +++ lucene/solr/trunk/src/java/org/apache/solr/search/WildcardFilter.java 
 Thu Aug 28 13:54:24 2008
 : @@ -0,0 +1,103 @@
 : +/**
 : + * Licensed to the Apache Software Foundation (ASF) under one or more
 : + * contributor license agreements.  See the NOTICE file distributed with
 : + * this work for additional information regarding copyright ownership.
 : + * The ASF licenses this file to You under the Apache License, Version 2.0
 : + * (the License); you may

Re: svn commit: r689978 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/search/SolrQueryParser.java src/java/org/apache/solr/search/WildcardFilter.java src/test/org/apache/solr/Converted

[Chris Hostetter]

: fire.  Imagine everything working fine for months, docs being
: occasionally added, until *boom* a magic limit is hit.
: 
: I wouldn't be opposed to a config option I guess... but I think the
: default should be something that doesn't unpredictably break (so it
: still wouldn't be 100% back compatible).

I don't disagree with you (ok, i might disagree with you slightly, but 
only on the semantics of what default could mean)  I'm just saying: this 
change will break highlighting for people where it currently works, and 
there won't be anything they can do to make it work the way it use to.

Yes, it can be a timebomb for some people, but for others with small 
indexes and a managable umber of terms it works just fine.

-Hoss

Re: svn commit: r689949 - /lucene/solr/branches/branch-1.3/CHANGES.txt

[Chris Hostetter]

I don't think we want the Release 1.4-dev stuff at the top of the 
CHANGES.txt included in the 1.3 release.

should that just added to the trunk copy?



On Thu, 28 Aug 2008 [EMAIL PROTECTED] wrote:

: Date: Thu, 28 Aug 2008 19:41:57 -
: From: [EMAIL PROTECTED]
: Reply-To: solr-dev@lucene.apache.org
: To: [EMAIL PROTECTED]
: Subject: svn commit: r689949 - /lucene/solr/branches/branch-1.3/CHANGES.txt
: 
: Author: gsingers
: Date: Thu Aug 28 12:41:57 2008
: New Revision: 689949
: 
: URL: http://svn.apache.org/viewvc?rev=689949view=rev
: Log:
: update changes for release
: 
: Modified:
: lucene/solr/branches/branch-1.3/CHANGES.txt
: 
: Modified: lucene/solr/branches/branch-1.3/CHANGES.txt
: URL: 
http://svn.apache.org/viewvc/lucene/solr/branches/branch-1.3/CHANGES.txt?rev=689949r1=689948r2=689949view=diff
: ==
: --- lucene/solr/branches/branch-1.3/CHANGES.txt (original)
: +++ lucene/solr/branches/branch-1.3/CHANGES.txt Thu Aug 28 12:41:57 2008
: @@ -20,8 +20,34 @@
:  
:  
:  $Id$
: +== Release 1.4-dev ==
: +Upgrading from Solr 1.3
: +---
: +
: +Detailed Change List
: +--
: +
: +New Features
: +--
: +
: +Optimizations
: +--
: +
: +Bug Fixes
: +--
: +
: +
: +Build
: +--
: +
: +
: +Documentation
: +--
: +
: +
: +
: +== Release 1.3.0 20080903 ==
:  
: -== Release 1.3-dev ==
:  
:  Upgrading from Solr 1.2
:  ---
: 
: 



-Hoss

Re: CHANGES.txt

[Erik Hatcher]

On Aug 16, 2008, at 10:52 PM, Koji Sekiguchi wrote:
BTW, latest solr-ruby version is v0.0.6. How about v1.0 or v1.3 when  
solr 1.3 is released?


I don't have any strong opinion on that.  Since solr-ruby will ship  
(it gets packaged, right?) with Solr 1.3 distribution, the version  
that it ships with should be considered the official one for that  
release.


How Apache releases and rubyforge gem releases sync I'm not yet  
sure how we should do that.


Unfortunately I haven't been doing very much Ruby lately.  The active  
folks in solr-ruby get to make the decisions... so whatever you feel  
is best I'm fine with for now. :)


Erik

Re: CHANGES.txt

[Koji Sekiguchi]

For the solr-ruby version, I'll take a poll on solr-user and ruby-dev as 
soon as I pick up options I'm thinking now.


Koji


Erik Hatcher wrote:


On Aug 16, 2008, at 10:52 PM, Koji Sekiguchi wrote:
BTW, latest solr-ruby version is v0.0.6. How about v1.0 or v1.3 when 
solr 1.3 is released?


I don't have any strong opinion on that.  Since solr-ruby will ship 
(it gets packaged, right?) with Solr 1.3 distribution, the version 
that it ships with should be considered the official one for that 
release.


How Apache releases and rubyforge gem releases sync I'm not yet 
sure how we should do that.


Unfortunately I haven't been doing very much Ruby lately.  The active 
folks in solr-ruby get to make the decisions... so whatever you feel 
is best I'm fine with for now. :)


Erik

Re: CHANGES.txt

[Koji Sekiguchi]

I've created client/ruby/CHANGES.txt to care for both solr-ruby and flare.

Koji

Erik Hatcher wrote:


On Aug 14, 2008, at 10:40 PM, Koji Sekiguchi wrote:


 the ruby client has its own changelog.

Oops. I wasn't aware of it's existence. But it's name is CHANGES.yml, 
not CHANGES.txt,

and the contents look like gem update info maintained by Erik.

If Solrj and DIH have their own CHANGES.txt, should solr-ruby have 
its CHANGES.txt?


CHANGES.yml was just me being different and having a structured usable 
change log, just showing off.  it's manually maintained and isn't 
being used for gems per se (other than manually copying/pasting the 
log over into rubyforge's form when pushing a new gem).


i dunno, i kinda like CHANGES.yml myself, but i'm not doing near as 
much Ruby as I'd like these days so whoever has a scratch to itch down 
there, go for it.


Erik

Re: CHANGES.txt

[Erik Hatcher]

On Aug 16, 2008, at 7:45 AM, Koji Sekiguchi wrote:
I've created client/ruby/CHANGES.txt to care for both solr-ruby and  
flare.


What?!  No rake task to generate it from the YAML?  ;)

just teasin...

Erik

Re: CHANGES.txt

[Koji Sekiguchi]

 I've created client/ruby/CHANGES.txt to care for both solr-ruby and 
flare.

 What?!  No rake task to generate it from the YAML?  ;)

After all, I'd like to maintain client/ruby/solr-ruby/CHANGES.yml rather 
than client/ruby/CHANGES.txt.

Now I delete CHANGES.txt:

$ svn commit
Deleting   client/ruby/CHANGES.txt
Sendingclient/ruby/solr-ruby

Committed revision 686583.

BTW, latest solr-ruby version is v0.0.6. How about v1.0 or v1.3 when 
solr 1.3 is released?


Koji

Re: CHANGES.txt

[Erik Hatcher]

On Aug 14, 2008, at 10:40 PM, Koji Sekiguchi wrote:


 the ruby client has its own changelog.

Oops. I wasn't aware of it's existence. But it's name is  
CHANGES.yml, not CHANGES.txt,

and the contents look like gem update info maintained by Erik.

If Solrj and DIH have their own CHANGES.txt, should solr-ruby have  
its CHANGES.txt?


CHANGES.yml was just me being different and having a structured usable  
change log, just showing off.  it's manually maintained and isn't  
being used for gems per se (other than manually copying/pasting the  
log over into rubyforge's form when pushing a new gem).


i dunno, i kinda like CHANGES.yml myself, but i'm not doing near as  
much Ruby as I'd like these days so whoever has a scratch to itch down  
there, go for it.


Erik

Re: CHANGES.txt

[Shalin Shekhar Mangar]

On a related note, I can see a CHANGES.txt inside client/java/solrj folder.
However, it has not been updated with many changes that have took place in
Solrj. I wasn't even aware of it's existence until recently.

Two options:
1. We merge it with the main changelog file and delete it
2. We gather all SolrJ changes from the main changelog and put it here

If we decide to go for the 2nd route, is a separate changelog needed for DIH
as well?

On Sat, Aug 9, 2008 at 8:29 AM, Ryan McKinley [EMAIL PROTECTED] wrote:

 Hello-

 Otis pointed out (via direct email) that I have not added many notes in
 CHANGES.txt recently.  I may be getting slopy, but I also remember some
 discussion about how to use changes.txt a while back and figure we should
 revist it to make sure everything necessary is in there before 1.3.

 My understanding is that CHANGES.txt is for folks following what has
 happened between releases -- it does not need to include internal
 modifications if there is no impact on the user.

 For example, SOLR-493 is not there since it fixes an issue introduced since
 1.2 -- using CHANGES.txt to say what happened in SOLR-142 had some
 side-effects that we now have cleared up seems overly complicated for anyone
 following.  Likewise the recent changes/modifictions to the
 UpdateRequestProcessor framework.

 Is this understanding correct?

 Are there more (or fewer) things we should include in CHANGES.txt?

 ryan




-- 
Regards,
Shalin Shekhar Mangar.

Re: CHANGES.txt

[Otis Gospodnetic]

I wasn't aware of it either :)
I like the idea of keeping it separate, mostly because I fear the core 
CHANGES.txt growing super big and thus harder for people to use, especially as 
contrib/ starts growing.

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



- Original Message 
 From: Shalin Shekhar Mangar [EMAIL PROTECTED]
 To: solr-dev@lucene.apache.org
 Sent: Thursday, August 14, 2008 4:27:22 PM
 Subject: Re: CHANGES.txt
 
 On a related note, I can see a CHANGES.txt inside client/java/solrj folder.
 However, it has not been updated with many changes that have took place in
 Solrj. I wasn't even aware of it's existence until recently.
 
 Two options:
 1. We merge it with the main changelog file and delete it
 2. We gather all SolrJ changes from the main changelog and put it here
 
 If we decide to go for the 2nd route, is a separate changelog needed for DIH
 as well?
 
 On Sat, Aug 9, 2008 at 8:29 AM, Ryan McKinley wrote:
 
  Hello-
 
  Otis pointed out (via direct email) that I have not added many notes in
  CHANGES.txt recently.  I may be getting slopy, but I also remember some
  discussion about how to use changes.txt a while back and figure we should
  revist it to make sure everything necessary is in there before 1.3.
 
  My understanding is that CHANGES.txt is for folks following what has
  happened between releases -- it does not need to include internal
  modifications if there is no impact on the user.
 
  For example, SOLR-493 is not there since it fixes an issue introduced since
  1.2 -- using CHANGES.txt to say what happened in SOLR-142 had some
  side-effects that we now have cleared up seems overly complicated for anyone
  following.  Likewise the recent changes/modifictions to the
  UpdateRequestProcessor framework.
 
  Is this understanding correct?
 
  Are there more (or fewer) things we should include in CHANGES.txt?
 
  ryan
 
 
 
 
 -- 
 Regards,
 Shalin Shekhar Mangar.

Re: CHANGES.txt

[Ryan McKinley]

I think an independent location for solrj changes is a good thing.   
They are not relevant for anyone using any other client...  the ruby  
client has its own changelog.  Solrj is a bit strange though since it  
is used within the SearchComponents.


Since solrj has not yet been released, the coverage in CHANGES.txt it  
a bit limited -- i don't think it is important for to follow all the  
internal changes.


Re independent change log for DIH?  that sounds good to me.

ryan


On Aug 14, 2008, at 4:27 PM, Shalin Shekhar Mangar wrote:

On a related note, I can see a CHANGES.txt inside client/java/solrj  
folder.
However, it has not been updated with many changes that have took  
place in

Solrj. I wasn't even aware of it's existence until recently.

Two options:
1. We merge it with the main changelog file and delete it
2. We gather all SolrJ changes from the main changelog and put it here

If we decide to go for the 2nd route, is a separate changelog needed  
for DIH

as well?

On Sat, Aug 9, 2008 at 8:29 AM, Ryan McKinley [EMAIL PROTECTED]  
wrote:



Hello-

Otis pointed out (via direct email) that I have not added many  
notes in
CHANGES.txt recently.  I may be getting slopy, but I also remember  
some
discussion about how to use changes.txt a while back and figure we  
should

revist it to make sure everything necessary is in there before 1.3.

My understanding is that CHANGES.txt is for folks following what has
happened between releases -- it does not need to include internal
modifications if there is no impact on the user.

For example, SOLR-493 is not there since it fixes an issue  
introduced since

1.2 -- using CHANGES.txt to say what happened in SOLR-142 had some
side-effects that we now have cleared up seems overly complicated  
for anyone

following.  Likewise the recent changes/modifictions to the
UpdateRequestProcessor framework.

Is this understanding correct?

Are there more (or fewer) things we should include in CHANGES.txt?

ryan





--
Regards,
Shalin Shekhar Mangar.

Re: CHANGES.txt

[Shalin Shekhar Mangar]

Ok, just to summarize: We will maintain separate changelogs for both SolrJ
and DIH. We should also make sure these changelogs are included in the
nightly tarballs.

Even if they are internal changes, no harm in documenting them in a single
place. Besides it is the only place where we give credit to our contributors
in a proper fashion.

So should we move all SolrJ and DIH related changes to their respective
changelogs? I share Otis's concern on the inflating main changelog file.

On Fri, Aug 15, 2008 at 2:11 AM, Ryan McKinley [EMAIL PROTECTED] wrote:

 I think an independent location for solrj changes is a good thing.  They
 are not relevant for anyone using any other client...  the ruby client has
 its own changelog.  Solrj is a bit strange though since it is used within
 the SearchComponents.

 Since solrj has not yet been released, the coverage in CHANGES.txt it a bit
 limited -- i don't think it is important for to follow all the internal
 changes.

 Re independent change log for DIH?  that sounds good to me.

 ryan



-- 
Regards,
Shalin Shekhar Mangar.

Re: CHANGES.txt

[Otis Gospodnetic]

I wouldn't worry about moving of already recorded changes (i.e. cleanup).  I'd 
just do the right thing from now on.

Getting 1.3 out next week sounds much more interesting to me. :)

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



- Original Message 
 From: Shalin Shekhar Mangar [EMAIL PROTECTED]
 To: solr-dev@lucene.apache.org
 Sent: Thursday, August 14, 2008 5:16:07 PM
 Subject: Re: CHANGES.txt
 
 Ok, just to summarize: We will maintain separate changelogs for both SolrJ
 and DIH. We should also make sure these changelogs are included in the
 nightly tarballs.
 
 Even if they are internal changes, no harm in documenting them in a single
 place. Besides it is the only place where we give credit to our contributors
 in a proper fashion.
 
 So should we move all SolrJ and DIH related changes to their respective
 changelogs? I share Otis's concern on the inflating main changelog file.
 
 On Fri, Aug 15, 2008 at 2:11 AM, Ryan McKinley wrote:
 
  I think an independent location for solrj changes is a good thing.  They
  are not relevant for anyone using any other client...  the ruby client has
  its own changelog.  Solrj is a bit strange though since it is used within
  the SearchComponents.
 
  Since solrj has not yet been released, the coverage in CHANGES.txt it a bit
  limited -- i don't think it is important for to follow all the internal
  changes.
 
  Re independent change log for DIH?  that sounds good to me.
 
  ryan
 
 
 
 -- 
 Regards,
 Shalin Shekhar Mangar.

Re: CHANGES.txt

[Koji Sekiguchi]

 the ruby client has its own changelog.

Oops. I wasn't aware of it's existence. But it's name is CHANGES.yml, 
not CHANGES.txt,

and the contents look like gem update info maintained by Erik.

If Solrj and DIH have their own CHANGES.txt, should solr-ruby have its 
CHANGES.txt?


Koji


Ryan McKinley wrote:
I think an independent location for solrj changes is a good thing.  
They are not relevant for anyone using any other client...  the ruby 
client has its own changelog.  Solrj is a bit strange though since it 
is used within the SearchComponents.


Since solrj has not yet been released, the coverage in CHANGES.txt it 
a bit limited -- i don't think it is important for to follow all the 
internal changes.


Re independent change log for DIH?  that sounds good to me.

ryan

Re: CHANGES.txt

[Chris Hostetter]

: My understanding is that CHANGES.txt is for folks following what has happened
: between releases -- it does not need to include internal modifications if
: there is no impact on the user.

the one thing to keep in mind is that people trying out nightly builds 
also take advantage of CHANGES.txt to know what to expect ... they can 
test the fix, or pay attention to wether the fix had some side affect on 
the functionality, etc...

: Are there more (or fewer) things we should include in CHANGES.txt?

In the specific case of SOLR-493, it would probably make sense to add a 
note to the CHANGES.txt entry for SOLR-142 saying that if there is a 
gettableFiles section an instance of ShowFileRequestHandler will be 
registered automaticly.



-Hoss

CHANGES.txt

[Ryan McKinley]

Hello-

Otis pointed out (via direct email) that I have not added many notes  
in CHANGES.txt recently.  I may be getting slopy, but I also remember  
some discussion about how to use changes.txt a while back and figure  
we should revist it to make sure everything necessary is in there  
before 1.3.


My understanding is that CHANGES.txt is for folks following what has  
happened between releases -- it does not need to include internal  
modifications if there is no impact on the user.


For example, SOLR-493 is not there since it fixes an issue introduced  
since 1.2 -- using CHANGES.txt to say what happened in SOLR-142 had  
some side-effects that we now have cleared up seems overly complicated  
for anyone following.  Likewise the recent changes/modifictions to the  
UpdateRequestProcessor framework.


Is this understanding correct?

Are there more (or fewer) things we should include in CHANGES.txt?

ryan

Re: CHANGES.txt

[Yonik Seeley]

On Fri, Aug 8, 2008 at 10:59 PM, Ryan McKinley [EMAIL PROTECTED] wrote:
 My understanding is that CHANGES.txt is for folks following what has
 happened between releases -- it does not need to include internal
 modifications if there is no impact on the user.

Correct.
I sometimes add an entry anyway if the changes are non-trivial and by
someone who hasn't received much credit yet.  If it fits under a
previous entry, I've also just added their name to the list of
contributors for that feature.

-Yonik

Re: svn commit: r659664 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/common/params/HighlightParams.java src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java src/test/org/apa

[Grant Ingersoll]

Well, if we're not sure on upgrading, we probably should roll this one  
back.  I personally haven't tested 2.4-dev with Solr, but it probably  
is fine.  I don't think copying just the highlighter is the right thing.


-Grant

On May 23, 2008, at 11:15 PM, Otis Gospodnetic wrote:

Ah, sorry about that - I was looking at that issue trying to  
remember why I made a reference to Lucene.  Now I remember. :)


I assume nobody minds getting 2.4-dev in there, right?  Shall I move  
all lib/lucene*jar to 2.4-dev or just the highlighter?



Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch


- Original Message 

From: Grant Ingersoll [EMAIL PROTECTED]
To: solr-dev@lucene.apache.org
Sent: Friday, May 23, 2008 6:47:28 PM
Subject: Re: svn commit: r659664 - in /lucene/solr/trunk:  
CHANGES.txt src/java/org/apache/solr/common/params/ 
HighlightParams.java src/java/org/apache/solr/highlight/ 
DefaultSolrHighlighter.java src/test/org/apache/solr/highlight/ 
HighlighterTest.java


I'm getting compile errors on clean :
 [mkdir] Created dir:/solr-trunk/build/core
[javac] Compiling 314 source files to ...solr-trunk/build/core
[javac] ...solr-trunk/src/java/org/apache/solr/highlight/
DefaultSolrHighlighter.java:45: cannot find symbol
[javac] symbol  : class SpanScorer
[javac] location: package org.apache.lucene.search.highlight
[javac] import org.apache.lucene.search.highlight.SpanScorer;
[javac]   ^
[javac] ...solr-trunk/src/java/org/apache/solr/highlight/
DefaultSolrHighlighter.java:144: cannot find symbol
[javac] symbol  : class SpanScorer
[javac] location: class
org.apache.solr.highlight.DefaultSolrHighlighter
[javac]   private SpanScorer getSpanQueryScorer(Query query,
String fieldName, CachingTokenFilter tokenStream, SolrQueryRequest
request) throws IOException {
[javac]   ^
[javac] ...solr-trunk/src/java/org/apache/solr/highlight/
DefaultSolrHighlighter.java:147: cannot find symbol
[javac] symbol  : class SpanScorer
[javac] location: class
org.apache.solr.highlight.DefaultSolrHighlighter
[javac]   return new SpanScorer(query, fieldName,  
tokenStream);

[javac]  ^
[javac] ...solr-trunk/src/java/org/apache/solr/highlight/
DefaultSolrHighlighter.java:150: cannot find symbol
[javac] symbol  : class SpanScorer
[javac] location: class
org.apache.solr.highlight.DefaultSolrHighlighter
[javac]   return new SpanScorer(query, null, tokenStream);
[javac]  ^
[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] Note: Some input files use unchecked or unsafe  
operations.

[javac] Note: Recompile with -Xlint:unchecked for details.
[javac] 4 errors

SVN Info:


svn info

Path: .
URL: https://svn.apache.org/repos/asf/lucene/solr/trunk
Repository Root: https://svn.apache.org/repos/asf
Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68
Revision: 659696
Node Kind: directory
Schedule: normal
Last Changed Author: otis
Last Changed Rev: 659668
Last Changed Date: 2008-05-23 17:32:45 -0400 (Fri, 23 May 2008)

Doesn't this require new Lucene jars?

-Grant



On May 23, 2008, at 5:23 PM, [EMAIL PROTECTED] wrote:


Author: otis
Date: Fri May 23 14:23:25 2008
New Revision: 659664

URL: http://svn.apache.org/viewvc?rev=659664view=rev
Log:
SOLR-553 Use SpanScorer when highlighting phrase terms and
hl.usePhraseHighlighter=true

Modified:
  lucene/solr/trunk/CHANGES.txt
  lucene/solr/trunk/src/java/org/apache/solr/common/params/
HighlightParams.java
  lucene/solr/trunk/src/java/org/apache/solr/highlight/
DefaultSolrHighlighter.java
  lucene/solr/trunk/src/test/org/apache/solr/highlight/
HighlighterTest.java

Modified: lucene/solr/trunk/CHANGES.txt
URL:

http://svn.apache.org/viewvc/lucene/solr/trunk/CHANGES.txt?rev=659664r1=659663r2=659664view=diff

=
=
=
=
=
=
=
=
= 
= 


--- lucene/solr/trunk/CHANGES.txt (original)
+++ lucene/solr/trunk/CHANGES.txt Fri May 23 14:23:25 2008
@@ -409,7 +409,14 @@

31. SOLR-514: Added explicit media-type with UTF* charset to *.xsl
files that
   don't already have one. (hossman)
-
+
+32. SOLR-505: Give RequestHandlers the possiblity to suppress the
generation
+of HTTP caching headers.  (Thomas Peuss via Otis Gospodnetic)
+
+33. SOLR-553: Handle highlighting of phrase terms better when
+hl.usePhraseHighligher=true URL param is used.
+(Bojan Smid via Otis Gospodnetic)
+
Other Changes
1. SOLR-135: Moved common classes to org.apache.solr.common and
altered the
   build scripts to make two jars: apache-solr-1.3.jar and

Modified: lucene/solr/trunk/src/java/org/apache/solr/common/params/
HighlightParams.java
URL:

http://svn.apache.org/viewvc/lucene/solr/trunk/src/java/org/apache/solr/common/params/HighlightParams.java?rev=659664r1=659663r2=659664view

Re: svn commit: r659664 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/common/params/HighlightParams.java src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java src/test/org/apa

[Grant Ingersoll]

So, upgrading to Lucene trunk breaks the SpellCheckerRequestHandler:

 Running org.apache.solr.handler.SpellCheckerRequestHandlerTest
[junit] response
[junit] responseHeaderstatus0/statusQTime1/QTime/ 
responseHeaderint name=numDocs10/intlst name=resultlst  
name=catint name=frequency1/intlst name=suggestionslst  
name=catint name=frequency1/int/lst/lst/lst/lst

[junit] /response
[junit] )
[junit] Tests run: 5, Failures: 1, Errors: 0, Time elapsed: 3.037  
sec
[junit] Test  
org.apache.solr.handler.SpellCheckerRequestHandlerTest FAILED



I only see a few changes in Lucene that were related to the Spell  
Checker:

https://issues.apache.org/jira/browse/LUCENE-1046
https://issues.apache.org/jira/browse/LUCENE-852
https://issues.apache.org/jira/browse/LUCENE-1152

Frankly, the only one that seems like it would be the cause is 1046,  
which means the test is probably wrong.


I'd suggest we rollback this change until proper testing has occurred.

-Grant


On May 24, 2008, at 7:02 AM, Grant Ingersoll wrote:

Well, if we're not sure on upgrading, we probably should roll this  
one back.  I personally haven't tested 2.4-dev with Solr, but it  
probably is fine.  I don't think copying just the highlighter is the  
right thing.


-Grant

On May 23, 2008, at 11:15 PM, Otis Gospodnetic wrote:

Ah, sorry about that - I was looking at that issue trying to  
remember why I made a reference to Lucene.  Now I remember. :)


I assume nobody minds getting 2.4-dev in there, right?  Shall I  
move all lib/lucene*jar to 2.4-dev or just the highlighter?



Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch


- Original Message 

From: Grant Ingersoll [EMAIL PROTECTED]
To: solr-dev@lucene.apache.org
Sent: Friday, May 23, 2008 6:47:28 PM
Subject: Re: svn commit: r659664 - in /lucene/solr/trunk:  
CHANGES.txt src/java/org/apache/solr/common/params/ 
HighlightParams.java src/java/org/apache/solr/highlight/ 
DefaultSolrHighlighter.java src/test/org/apache/solr/highlight/ 
HighlighterTest.java


I'm getting compile errors on clean :
[mkdir] Created dir:/solr-trunk/build/core
   [javac] Compiling 314 source files to ...solr-trunk/build/core
   [javac] ...solr-trunk/src/java/org/apache/solr/highlight/
DefaultSolrHighlighter.java:45: cannot find symbol
   [javac] symbol  : class SpanScorer
   [javac] location: package org.apache.lucene.search.highlight
   [javac] import org.apache.lucene.search.highlight.SpanScorer;
   [javac]   ^
   [javac] ...solr-trunk/src/java/org/apache/solr/highlight/
DefaultSolrHighlighter.java:144: cannot find symbol
   [javac] symbol  : class SpanScorer
   [javac] location: class
org.apache.solr.highlight.DefaultSolrHighlighter
   [javac]   private SpanScorer getSpanQueryScorer(Query query,
String fieldName, CachingTokenFilter tokenStream, SolrQueryRequest
request) throws IOException {
   [javac]   ^
   [javac] ...solr-trunk/src/java/org/apache/solr/highlight/
DefaultSolrHighlighter.java:147: cannot find symbol
   [javac] symbol  : class SpanScorer
   [javac] location: class
org.apache.solr.highlight.DefaultSolrHighlighter
   [javac]   return new SpanScorer(query, fieldName,  
tokenStream);

   [javac]  ^
   [javac] ...solr-trunk/src/java/org/apache/solr/highlight/
DefaultSolrHighlighter.java:150: cannot find symbol
   [javac] symbol  : class SpanScorer
   [javac] location: class
org.apache.solr.highlight.DefaultSolrHighlighter
   [javac]   return new SpanScorer(query, null, tokenStream);
   [javac]  ^
   [javac] Note: Some input files use or override a deprecated API.
   [javac] Note: Recompile with -Xlint:deprecation for details.
   [javac] Note: Some input files use unchecked or unsafe  
operations.

   [javac] Note: Recompile with -Xlint:unchecked for details.
   [javac] 4 errors

SVN Info:


svn info

Path: .
URL: https://svn.apache.org/repos/asf/lucene/solr/trunk
Repository Root: https://svn.apache.org/repos/asf
Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68
Revision: 659696
Node Kind: directory
Schedule: normal
Last Changed Author: otis
Last Changed Rev: 659668
Last Changed Date: 2008-05-23 17:32:45 -0400 (Fri, 23 May 2008)

Doesn't this require new Lucene jars?

-Grant



On May 23, 2008, at 5:23 PM, [EMAIL PROTECTED] wrote:


Author: otis
Date: Fri May 23 14:23:25 2008
New Revision: 659664

URL: http://svn.apache.org/viewvc?rev=659664view=rev
Log:
SOLR-553 Use SpanScorer when highlighting phrase terms and
hl.usePhraseHighlighter=true

Modified:
 lucene/solr/trunk/CHANGES.txt
 lucene/solr/trunk/src/java/org/apache/solr/common/params/
HighlightParams.java
 lucene/solr/trunk/src/java/org/apache/solr/highlight/
DefaultSolrHighlighter.java
 lucene/solr/trunk/src/test/org/apache/solr/highlight/
HighlighterTest.java

Modified: lucene/solr/trunk/CHANGES.txt
URL:

http://svn.apache.org/viewvc/lucene/solr/trunk/CHANGES.txt?rev=659664r1=659663r2=659664view=diff

Re: svn commit: r659664 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/common/params/HighlightParams.java src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java src/test/org/apa

[Grant Ingersoll]

Also https://issues.apache.org/jira/browse/LUCENE-1183


On May 24, 2008, at 8:06 AM, Grant Ingersoll wrote:


So, upgrading to Lucene trunk breaks the SpellCheckerRequestHandler:

Running org.apache.solr.handler.SpellCheckerRequestHandlerTest
   [junit] response
   [junit] responseHeaderstatus0/statusQTime1/QTime/ 
responseHeaderint name=numDocs10/intlst name=resultlst  
name=catint name=frequency1/intlst name=suggestionslst  
name=catint name=frequency1/int/lst/lst/lst/lst

   [junit] /response
   [junit] )
   [junit] Tests run: 5, Failures: 1, Errors: 0, Time elapsed: 3.037  
sec
   [junit] Test  
org.apache.solr.handler.SpellCheckerRequestHandlerTest FAILED



I only see a few changes in Lucene that were related to the Spell  
Checker:

https://issues.apache.org/jira/browse/LUCENE-1046
https://issues.apache.org/jira/browse/LUCENE-852
https://issues.apache.org/jira/browse/LUCENE-1152

Frankly, the only one that seems like it would be the cause is 1046,  
which means the test is probably wrong.


I'd suggest we rollback this change until proper testing has occurred.

-Grant


On May 24, 2008, at 7:02 AM, Grant Ingersoll wrote:

Well, if we're not sure on upgrading, we probably should roll this  
one back.  I personally haven't tested 2.4-dev with Solr, but it  
probably is fine.  I don't think copying just the highlighter is  
the right thing.


-Grant

On May 23, 2008, at 11:15 PM, Otis Gospodnetic wrote:

Ah, sorry about that - I was looking at that issue trying to  
remember why I made a reference to Lucene.  Now I remember. :)


I assume nobody minds getting 2.4-dev in there, right?  Shall I  
move all lib/lucene*jar to 2.4-dev or just the highlighter?



Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch


- Original Message 

From: Grant Ingersoll [EMAIL PROTECTED]
To: solr-dev@lucene.apache.org
Sent: Friday, May 23, 2008 6:47:28 PM
Subject: Re: svn commit: r659664 - in /lucene/solr/trunk:  
CHANGES.txt src/java/org/apache/solr/common/params/ 
HighlightParams.java src/java/org/apache/solr/highlight/ 
DefaultSolrHighlighter.java src/test/org/apache/solr/highlight/ 
HighlighterTest.java


I'm getting compile errors on clean :
[mkdir] Created dir:/solr-trunk/build/core
  [javac] Compiling 314 source files to ...solr-trunk/build/core
  [javac] ...solr-trunk/src/java/org/apache/solr/highlight/
DefaultSolrHighlighter.java:45: cannot find symbol
  [javac] symbol  : class SpanScorer
  [javac] location: package org.apache.lucene.search.highlight
  [javac] import org.apache.lucene.search.highlight.SpanScorer;
  [javac]   ^
  [javac] ...solr-trunk/src/java/org/apache/solr/highlight/
DefaultSolrHighlighter.java:144: cannot find symbol
  [javac] symbol  : class SpanScorer
  [javac] location: class
org.apache.solr.highlight.DefaultSolrHighlighter
  [javac]   private SpanScorer getSpanQueryScorer(Query query,
String fieldName, CachingTokenFilter tokenStream, SolrQueryRequest
request) throws IOException {
  [javac]   ^
  [javac] ...solr-trunk/src/java/org/apache/solr/highlight/
DefaultSolrHighlighter.java:147: cannot find symbol
  [javac] symbol  : class SpanScorer
  [javac] location: class
org.apache.solr.highlight.DefaultSolrHighlighter
  [javac]   return new SpanScorer(query, fieldName,  
tokenStream);

  [javac]  ^
  [javac] ...solr-trunk/src/java/org/apache/solr/highlight/
DefaultSolrHighlighter.java:150: cannot find symbol
  [javac] symbol  : class SpanScorer
  [javac] location: class
org.apache.solr.highlight.DefaultSolrHighlighter
  [javac]   return new SpanScorer(query, null, tokenStream);
  [javac]  ^
  [javac] Note: Some input files use or override a deprecated API.
  [javac] Note: Recompile with -Xlint:deprecation for details.
  [javac] Note: Some input files use unchecked or unsafe  
operations.

  [javac] Note: Recompile with -Xlint:unchecked for details.
  [javac] 4 errors

SVN Info:


svn info

Path: .
URL: https://svn.apache.org/repos/asf/lucene/solr/trunk
Repository Root: https://svn.apache.org/repos/asf
Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68
Revision: 659696
Node Kind: directory
Schedule: normal
Last Changed Author: otis
Last Changed Rev: 659668
Last Changed Date: 2008-05-23 17:32:45 -0400 (Fri, 23 May 2008)

Doesn't this require new Lucene jars?

-Grant



On May 23, 2008, at 5:23 PM, [EMAIL PROTECTED] wrote:


Author: otis
Date: Fri May 23 14:23:25 2008
New Revision: 659664

URL: http://svn.apache.org/viewvc?rev=659664view=rev
Log:
SOLR-553 Use SpanScorer when highlighting phrase terms and
hl.usePhraseHighlighter=true

Modified:
lucene/solr/trunk/CHANGES.txt
lucene/solr/trunk/src/java/org/apache/solr/common/params/
HighlightParams.java
lucene/solr/trunk/src/java/org/apache/solr/highlight/
DefaultSolrHighlighter.java
lucene/solr/trunk/src/test/org/apache/solr/highlight/
HighlighterTest.java

Modified: lucene/solr/trunk/CHANGES.txt
URL:

http://svn.apache.org/viewvc

Re: svn commit: r659664 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/common/params/HighlightParams.java src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java src/test/org/apa

[Otis Gospodnetic]

Ah, sorry about that - I was looking at that issue trying to remember why I 
made a reference to Lucene.  Now I remember. :)

I assume nobody minds getting 2.4-dev in there, right?  Shall I move all 
lib/lucene*jar to 2.4-dev or just the highlighter?


Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch


- Original Message 
 From: Grant Ingersoll [EMAIL PROTECTED]
 To: solr-dev@lucene.apache.org
 Sent: Friday, May 23, 2008 6:47:28 PM
 Subject: Re: svn commit: r659664 - in /lucene/solr/trunk: CHANGES.txt 
 src/java/org/apache/solr/common/params/HighlightParams.java 
 src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java 
 src/test/org/apache/solr/highlight/HighlighterTest.java
 
 I'm getting compile errors on clean :
   [mkdir] Created dir:/solr-trunk/build/core
  [javac] Compiling 314 source files to ...solr-trunk/build/core
  [javac] ...solr-trunk/src/java/org/apache/solr/highlight/ 
 DefaultSolrHighlighter.java:45: cannot find symbol
  [javac] symbol  : class SpanScorer
  [javac] location: package org.apache.lucene.search.highlight
  [javac] import org.apache.lucene.search.highlight.SpanScorer;
  [javac]   ^
  [javac] ...solr-trunk/src/java/org/apache/solr/highlight/ 
 DefaultSolrHighlighter.java:144: cannot find symbol
  [javac] symbol  : class SpanScorer
  [javac] location: class  
 org.apache.solr.highlight.DefaultSolrHighlighter
  [javac]   private SpanScorer getSpanQueryScorer(Query query,  
 String fieldName, CachingTokenFilter tokenStream, SolrQueryRequest  
 request) throws IOException {
  [javac]   ^
  [javac] ...solr-trunk/src/java/org/apache/solr/highlight/ 
 DefaultSolrHighlighter.java:147: cannot find symbol
  [javac] symbol  : class SpanScorer
  [javac] location: class  
 org.apache.solr.highlight.DefaultSolrHighlighter
  [javac]   return new SpanScorer(query, fieldName, tokenStream);
  [javac]  ^
  [javac] ...solr-trunk/src/java/org/apache/solr/highlight/ 
 DefaultSolrHighlighter.java:150: cannot find symbol
  [javac] symbol  : class SpanScorer
  [javac] location: class  
 org.apache.solr.highlight.DefaultSolrHighlighter
  [javac]   return new SpanScorer(query, null, tokenStream);
  [javac]  ^
  [javac] Note: Some input files use or override a deprecated API.
  [javac] Note: Recompile with -Xlint:deprecation for details.
  [javac] Note: Some input files use unchecked or unsafe operations.
  [javac] Note: Recompile with -Xlint:unchecked for details.
  [javac] 4 errors
 
 SVN Info:
 
 svn info
 Path: .
 URL: https://svn.apache.org/repos/asf/lucene/solr/trunk
 Repository Root: https://svn.apache.org/repos/asf
 Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68
 Revision: 659696
 Node Kind: directory
 Schedule: normal
 Last Changed Author: otis
 Last Changed Rev: 659668
 Last Changed Date: 2008-05-23 17:32:45 -0400 (Fri, 23 May 2008)
 
 Doesn't this require new Lucene jars?
 
 -Grant
 
 
 
 On May 23, 2008, at 5:23 PM, [EMAIL PROTECTED] wrote:
 
  Author: otis
  Date: Fri May 23 14:23:25 2008
  New Revision: 659664
 
  URL: http://svn.apache.org/viewvc?rev=659664view=rev
  Log:
  SOLR-553 Use SpanScorer when highlighting phrase terms and  
  hl.usePhraseHighlighter=true
 
  Modified:
 lucene/solr/trunk/CHANGES.txt
 lucene/solr/trunk/src/java/org/apache/solr/common/params/ 
  HighlightParams.java
 lucene/solr/trunk/src/java/org/apache/solr/highlight/ 
  DefaultSolrHighlighter.java
 lucene/solr/trunk/src/test/org/apache/solr/highlight/ 
  HighlighterTest.java
 
  Modified: lucene/solr/trunk/CHANGES.txt
  URL: 
 http://svn.apache.org/viewvc/lucene/solr/trunk/CHANGES.txt?rev=659664r1=659663r2=659664view=diff
  = 
  = 
  = 
  = 
  = 
  = 
  = 
  = 
  ==
  --- lucene/solr/trunk/CHANGES.txt (original)
  +++ lucene/solr/trunk/CHANGES.txt Fri May 23 14:23:25 2008
  @@ -409,7 +409,14 @@
 
  31. SOLR-514: Added explicit media-type with UTF* charset to *.xsl  
  files that
  don't already have one. (hossman)
  -
  +
  +32. SOLR-505: Give RequestHandlers the possiblity to suppress the  
  generation
  +of HTTP caching headers.  (Thomas Peuss via Otis Gospodnetic)
  +
  +33. SOLR-553: Handle highlighting of phrase terms better when
  +hl.usePhraseHighligher=true URL param is used.
  +(Bojan Smid via Otis Gospodnetic)
  +
  Other Changes
   1. SOLR-135: Moved common classes to org.apache.solr.common and  
  altered the
  build scripts to make two jars: apache-solr-1.3.jar and
 
  Modified: lucene/solr/trunk/src/java/org/apache/solr/common/params/ 
  HighlightParams.java
  URL: 
 http://svn.apache.org/viewvc/lucene/solr/trunk/src/java/org/apache/solr/common/params/HighlightParams.java?rev=659664r1=659663r2=659664view=diff

Re: svn commit: r641573 - /lucene/solr/trunk/CHANGES.txt

[Yonik Seeley]

Perhaps use the 1.2 upgrading statement as a template?  It mentions
how to handle master/slave upgrading.

-Yonik

On Wed, Mar 26, 2008 at 4:59 PM,  [EMAIL PROTECTED] wrote:
 Author: ryan
  Date: Wed Mar 26 13:59:40 2008
  New Revision: 641573

  URL: http://svn.apache.org/viewvc?rev=641573view=rev
  Log:
  adding an upgrade note about the lucene format changes

  Modified:
 lucene/solr/trunk/CHANGES.txt

  Modified: lucene/solr/trunk/CHANGES.txt
  URL: 
 http://svn.apache.org/viewvc/lucene/solr/trunk/CHANGES.txt?rev=641573r1=641572r2=641573view=diff
  
 ==
  --- lucene/solr/trunk/CHANGES.txt (original)
  +++ lucene/solr/trunk/CHANGES.txt Wed Mar 26 13:59:40 2008
  @@ -26,6 +26,12 @@
   Upgrading from Solr 1.2
   ---

  +Solr 1.3 includes a newer version of Lucene (2.3.1).  The new index
  +format is not readable by older versions of Lucene. Existing indexes
  +will be automatically converted to the new format after the first
  +update command.  Be sure to backup your index before upgrading in
  +case you need to downgrade.
  +
   Solr now recognizes HTTP Request headers related to HTTP Caching (see
   RFC 2616 sec13) and will by default respond with 304 Not Modified
   when appropriate.  This should only affect users who access Solr via

Re: svn commit: r638484 - /lucene/solr/trunk/CHANGES.txt

[Yonik Seeley]

On Tue, Mar 18, 2008 at 2:03 PM,  [EMAIL PROTECTED] wrote:
  +Solr now recognizes HTTP Request headers related to HTTP Caching (see
  +RFC 2616 sec13) and will by default respond with 304 Not Modified
  +when appropriate.  This should only affect users who access Solr via
  +an HTTP Cache,

It affects browsers too.  I noticed that no new request was being
executed when I hit refresh in firefox.
I worked around it by adding a random arg like x=1234 to get it to re-execute.

-Yonik

Re: svn commit: r638484 - /lucene/solr/trunk/CHANGES.txt

[Erik Hatcher]

On Mar 18, 2008, at 3:04 PM, Yonik Seeley wrote:

On Tue, Mar 18, 2008 at 2:03 PM,  [EMAIL PROTECTED] wrote:
 +Solr now recognizes HTTP Request headers related to HTTP Caching  
(see

 +RFC 2616 sec13) and will by default respond with 304 Not Modified
 +when appropriate.  This should only affect users who access Solr  
via

 +an HTTP Cache,


It affects browsers too.  I noticed that no new request was being
executed when I hit refresh in firefox.
I worked around it by adding a random arg like x=1234 to get it to  
re-execute.


I experienced the same thing, but instead of a random argument you  
can hold down shift while clicking the reload button to have it make  
a fresh request.


Erik

Re: svn commit: r638484 - /lucene/solr/trunk/CHANGES.txt

[Chris Hostetter]

:   +Solr now recognizes HTTP Request headers related to HTTP Caching (see
:   +RFC 2616 sec13) and will by default respond with 304 Not Modified
:   +when appropriate.  This should only affect users who access Solr via
:   +an HTTP Cache,
: 
: It affects browsers too.  I noticed that no new request was being
: executed when I hit refresh in firefox.

Well, I would argue that paragraph is technically correct since you are 
accessing Solr via an HTTP Cache ... your browser cache.

But I'll update it to make special note of that.

-Hoss

what goes in CHANGES.txt

[Yonik Seeley]

I was browsing recent changes to CHANGES.txt, and some of them seem
perhaps a bit low-level to be there.  Here's my view:
http://www.nabble.com/Re%3A-Tika-Changelog-p11082404.html

IMO, changes in implementation detail don't belong in CHANGES.txt,
unless it's to give credit to a contributor for that change (which
committers don't need).

For example, the following change shouldn't matter to normal users:
SOLR-262: Added toObject( Fieldable ) to FieldType. [...]

-Yonik

Re: what goes in CHANGES.txt

[Ryan McKinley]

IMO, changes in implementation detail don't belong in CHANGES.txt,
unless it's to give credit to a contributor for that change (which
committers don't need).



With SOLR-292, it seems confusing to list a bug fix for something added 
since the last release.  But since it is from a new/useful contributor I 
put it in.  Is that in line with your thoughts?



For example, the following change shouldn't matter to normal users:
SOLR-262: Added toObject( Fieldable ) to FieldType. [...]



I'll remove SOLR-262

Re: what goes in CHANGES.txt

[Chris Hostetter]

: http://www.nabble.com/Re%3A-Tika-Changelog-p11082404.html
:
: IMO, changes in implementation detail don't belong in CHANGES.txt,
: unless it's to give credit to a contributor for that change (which
: committers don't need).
:
: For example, the following change shouldn't matter to normal users:
: SOLR-262: Added toObject( Fieldable ) to FieldType. [...]

i agree with teh sentiments you posted to the tika list, but we have to
keep in mind that the scope of users is broad ... that note about
SOLR-262 may not be important to users that only deal with Solr out of the
box, but it is important to advanced users writing their own plugins --
and should be noted in teh CHANGES.txt somewhere ... but i would agree
that the New Features section probably isn't the right place for it.

this seems like a perfect example of the type of thing that belongs in
Other Changes ... or perhaps a new section in CHANGES specificly for
plugin developers.  (Internal Changes maybe?)


-Hoss

Re: what goes in CHANGES.txt

[Yonik Seeley]

On 7/9/07, Chris Hostetter [EMAIL PROTECTED] wrote:

: http://www.nabble.com/Re%3A-Tika-Changelog-p11082404.html
:
: IMO, changes in implementation detail don't belong in CHANGES.txt,
: unless it's to give credit to a contributor for that change (which
: committers don't need).
:
: For example, the following change shouldn't matter to normal users:
: SOLR-262: Added toObject( Fieldable ) to FieldType. [...]

i agree with teh sentiments you posted to the tika list, but we have to
keep in mind that the scope of users is broad ... that note about
SOLR-262 may not be important to users that only deal with Solr out of the
box, but it is important to advanced users writing their own plugins --
and should be noted in teh CHANGES.txt somewhere


I disagree that it needs to be in CHANGES.txt... advanced expert users
will also use the source/javadoc (and creating ones own field types is
definitely advanced).  Every public class and method in solr isn't
equally part of the public API.

It's somewhat subjective, but if we changed the semantics of something
like IndexSearcher.getDocList(), that should definitely be in
CHANGES.txt
If we added a more obscure method that didn't exist before (like
getFirstMatch()), that wouldn't need to be added (it's noise to most
users, doesn't change existing functionality, not accessible w/o
writing Java code, and an advanced user can pull up the javadoc).


... but i would agree
that the New Features section probably isn't the right place for it.


Yes, either other or a new java API section

-Yonik

Re: what goes in CHANGES.txt

[Chris Hostetter]

: I disagree that it needs to be in CHANGES.txt... advanced expert users
: will also use the source/javadoc (and creating ones own field types is
: definitely advanced).  Every public class and method in solr isn't
: equally part of the public API.

true, but classes like FieldType are specificly noted as extension
points...
http://wiki.apache.org/solr/SolrPlugins

...in general, any situatuation where we support dynamic loading of
class/interface impls is a situation where changes to that class/interface
should be documented in the CHANGES.txt because it affets people providing
their own impl ... new methods on abstract base classes are particularly
important because people upgrading won't notice any compilation errors,
and may then wonder why some features (which may have worked in the past)
no longer work with their custom class.

the FieldType.toObject example is a perfect case of this ... without
drawing atention to the fact that it was added to support updateable
documents people with their own custom FieldTypes might get really
confused as to why updating docs doesn't work for them

: If we added a more obscure method that didn't exist before (like
: getFirstMatch()), that wouldn't need to be added (it's noise to most
: users, doesn't change existing functionality, not accessible w/o
: writing Java code, and an advanced user can pull up the javadoc).

I agree ... but only because SOlrIndexSearcher isn't a class we expect
advanced users to subclass so providing a new impl that they dont' know
about doesn't affect them ... but in cases like
FieldType, RequestHandlerBase, or BaseTokenFilterFactory, etc then new
methods added to those classes that other parts of Solr rely on for new
functionality (or a new methodology for previously existing functionality)
should be documented someplace advanced users can find them without
needing to diff the javadocs between releases.




-Hoss

Re: svn commit: r547438 - /lucene/solr/trunk/CHANGES.txt

[Chris Hostetter]

: + 6. SOLR-20: Added a java client interface with two implementations.  One
: +implementation uses commons httpclient to connect to solr via HTTP.  The
: +other connects to solr directly.  Check client/java/solrj.  This addition
: +also includes tests that start jetty and test a connection using the full
: +HTTP request cycle.  (Darren Erik Vengroff, Will Johnson, ryan)

Ryan: I'm getting 44 compilation erors from the compile-solrj-core target
... root issues seem to be...

  package javax.xml.stream does not exist

...what jar is this suppose to be comming from?



-Hoss

Re: svn commit: r541391 - in /lucene/solr/trunk: CHANGES.txt example/solr/conf/xslt/example_atom.xsl example/solr/conf/xslt/example_rss.xsl

[Otis Gospodnetic]

I'd slap versions to those 2 XSL files to immediately answer which version of 
Atom|RSS does this produce?

Otis
 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Simpy -- http://www.simpy.com/  -  Tag  -  Search  -  Share

- Original Message 
From: [EMAIL PROTECTED] [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Thursday, May 24, 2007 3:06:31 PM
Subject: svn commit: r541391 - in /lucene/solr/trunk: CHANGES.txt 
example/solr/conf/xslt/example_atom.xsl example/solr/conf/xslt/example_rss.xsl

Author: hossman
Date: Thu May 24 12:06:29 2007
New Revision: 541391

URL: http://svn.apache.org/viewvc?view=revrev=541391
Log:
SOLR-208: example XSLTs for RSS and Atom

Added:
lucene/solr/trunk/example/solr/conf/xslt/example_atom.xsl   (with props)
lucene/solr/trunk/example/solr/conf/xslt/example_rss.xsl   (with props)
Modified:
lucene/solr/trunk/CHANGES.txt

Modified: lucene/solr/trunk/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/lucene/solr/trunk/CHANGES.txt?view=diffrev=541391r1=541390r2=541391
==
--- lucene/solr/trunk/CHANGES.txt (original)
+++ lucene/solr/trunk/CHANGES.txt Thu May 24 12:06:29 2007
@@ -197,7 +197,12 @@
 33. SOLR-234: TrimFilter can update the Token's startOffset and endOffset 
 if updateOffsets=true.  By default the Token offsets are unchanged.
 (ryan)
-
+
+34. SOLR-208: new example_rss.xsl and example_atom.xsl to provide more
+examples for people about the Solr XML response format and how they
+can transform it to suit different needs.
+(Brian Whitman via hossman)
+
 Changes in runtime behavior
  1. Highlighting using DisMax will only pick up terms from the main 
 user query, not boost or filter queries (klaas).

Added: lucene/solr/trunk/example/solr/conf/xslt/example_atom.xsl
URL: 
http://svn.apache.org/viewvc/lucene/solr/trunk/example/solr/conf/xslt/example_atom.xsl?view=autorev=541391
==
--- lucene/solr/trunk/example/solr/conf/xslt/example_atom.xsl (added)
+++ lucene/solr/trunk/example/solr/conf/xslt/example_atom.xsl Thu May 24 
12:06:29 2007
@@ -0,0 +1,63 @@
+?xml version='1.0' encoding='UTF-8'?
+
+!-- 
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ --
+
+!-- 
+  Simple transform of Solr query results to Atom
+ --
+
+xsl:stylesheet version='1.0'
+xmlns:xsl='http://www.w3.org/1999/XSL/Transform'
+
+  xsl:output method=xml encoding=utf-8 /
+
+  xsl:template match='/'
+xsl:variable name=query select=response/[EMAIL 
PROTECTED]'responseHeader']/[EMAIL PROTECTED]'params']/[EMAIL PROTECTED]'q']/
+feed xmlns=http://www.w3.org/2005/Atom;;
+  titleExample Solr Atom Feed/title
+  subtitle
+   This has been formatted by the sample example_atom.xsl transform -
+   use your own XSLT to get a nicer Atom feed.
+  /subtitle
+  author
+nameApache Solr/name
+email[EMAIL PROTECTED]/email
+  /author
+  link rel=self type=application/atom+xml 
+
href=http://localhost:8983/solr/q={$query}amp;wt=xsltamp;tr=atom.xsl/
+  updated
+xsl:value-of select=response/result/doc[position()=1]/[EMAIL 
PROTECTED]'timestamp']/
+  /updated
+  idtag:localhost,2007:example/id
+  xsl:apply-templates select=response/result/doc/
+/feed
+  /xsl:template
+
+  !-- search results xslt --
+  xsl:template match=doc
+xsl:variable name=id select=[EMAIL PROTECTED]'id']/
+entry
+  titlexsl:value-of select=[EMAIL PROTECTED]'name']//title
+  link href=http://localhost:8983/solr/select?q={$id}/
+  idtag:localhost,2007:xsl:value-of select=$id//id
+  summaryxsl:value-of select=[EMAIL PROTECTED]'features']//summary
+  updatedxsl:value-of select=[EMAIL PROTECTED]'timestamp']//updated
+/entry
+  /xsl:template
+
+/xsl:stylesheet

Propchange: lucene/solr/trunk/example/solr/conf/xslt/example_atom.xsl
--
svn:eol-style = native

Propchange: lucene/solr/trunk/example/solr/conf/xslt/example_atom.xsl

Re: svn commit: r541391 - in /lucene/solr/trunk: CHANGES.txt example/solr/conf/xslt/example_atom.xsl example/solr/conf/xslt/example_rss.xsl

[Chris Hostetter]

: I'd slap versions to those 2 XSL files to immediately answer which
: version of Atom|RSS does this produce?

i'm comfortable calling the example_rss.xsl RSS, since most RSS
readers will know what do do with it, but i don't know that i'm
comfrotable calling it any specific version of RSS, people are more likely
to get irrate about claiming ot be a specific version if one little thing
is wrong then they are about not claiming to be anything in particular.

example.xsl says it outputs HTML but doesn't make any specific claims
about which version of (x)html ... not asserting any particular allegience
in a holy war is a good way to avoid getting slaughtered :)



-Hoss

Re: svn commit: r541391 - in /lucene/solr/trunk: CHANGES.txt example/solr/conf/xslt/example_atom.xsl example/solr/conf/xslt/example_rss.xsl

[Walter Underwood]

On 5/25/07 10:45 AM, Chris Hostetter [EMAIL PROTECTED] wrote:
 
 : I'd slap versions to those 2 XSL files to immediately answer which
 : version of Atom|RSS does this produce?
 
 i'm comfortable calling the example_rss.xsl RSS, since most RSS
 readers will know what do do with it, but i don't know that i'm
 comfrotable calling it any specific version of RSS, people are more likely
 to get irrate about claiming ot be a specific version if one little thing
 is wrong then they are about not claiming to be anything in particular.

Some versions of RSS are quite incompatible, so we MUST say what
version we are implementing. RSS 1.0 is completely different from
the 0.9 series and 2.0.

Atom doesn't have a version number, but RFC 4287 Atom is informally
called 1.0. 

wunder

Re: svn commit: r541391 - in /lucene/solr/trunk: CHANGES.txt example/solr/conf/xslt/example_atom.xsl example/solr/conf/xslt/example_rss.xsl

[Brian Whitman]

Some versions of RSS are quite incompatible, so we MUST say what
version we are implementing. RSS 1.0 is completely different from
the 0.9 series and 2.0.

Atom doesn't have a version number, but RFC 4287 Atom is informally
called 1.0.



The rss_example is supposed to be RSS 2.0.

The atom_example validated on feedvalidator.org as Atom 1.0, I assume  
that's the RFC 4287.

Re: svn commit: r519107 - in /lucene/solr/trunk: CHANGES.txt example/exampledocs/spellchecher.xml example/solr/conf/solrconfig.xml lib/lucene-spellchecker-2.2-dev.jar src/java/org/apache/solr/request/

[Ryan McKinley]

Should I really move the SpellCheckerRequestHandler to o.a.solr.handler 
instead?  I see request handlers in both handler and request packages.



There are two handlers in o.a.s.request:
DisMaxRequestHandler.java
StandardRequestHandler.java

all the other handlers are in o.a.s.handler.  Moving DisMax and
Standard is harder because people may already be extending them. (At
some point we may want deprecate those and move them to .handler)

As we add things like CSVUpdateRequestHandler, LukeRequestHandler,
SQLUpdateReqeustHandler it seems cleaner (to me) to keep them isolated
from the response writers, parameter parsing etc...

Re: svn commit: r519107 - in /lucene/solr/trunk: CHANGES.txt example/exampledocs/spellchecher.xml example/solr/conf/solrconfig.xml lib/lucene-spellchecker-2.2-dev.jar src/java/org/apache/solr/request/

[Ryan McKinley]

A few trivial comments on SpellCheckerRequestHandler:

1. Should it go in org.apache.solr.handler rather then
org.apache.solr.request?  As the number of special handlers
increases, it seems good to have them semi-isolated.
2. replace tabs with two spaces
3. The file needs the svn props set
4. getVersion() should return $Revision: 501512 $; rather then
SolrCore.version

nit pick nit pick nit pick.  otherwise looks good!

Re: svn commit: r519107 - in /lucene/solr/trunk: CHANGES.txt example/exampledocs/spellchecher.xml example/solr/conf/solrconfig.xml lib/lucene-spellchecker-2.2-dev.jar src/java/org/apache/solr/request/

[Erik Hatcher]

On Mar 16, 2007, at 3:28 PM, [EMAIL PROTECTED] wrote:


Author: otis
lucene/solr/trunk/example/exampledocs/spellchecher.xml


that should be spellchecker, not spellchecher.

Re: svn commit: r519107 - in /lucene/solr/trunk: CHANGES.txt example/exampledocs/spellchecher.xml example/solr/conf/solrconfig.xml lib/lucene-spellchecker-2.2-dev.jar src/java/org/apache/solr/request/

[Erik Hatcher]

On Mar 16, 2007, at 3:28 PM, [EMAIL PROTECTED] wrote:
+ str name=spellcheckerIndexDir/home/otis/dev/repos/lucene/ 
solr/trunk/example/solr/data/index/str


And this needs to be changed to something globally relevant.  What  
triggers the creation of this index?   With this part of the example,  
we ought to flesh it out so that it works for anyone trying out Solr,  
right?



+public void init(NamedList args) {
+super.init( args );
+spellcheckerIndexDir = invariants.get 
(spellcheckerIndexDir);

+try {
+spellChecker = new SpellChecker 
(FSDirectory.getDirectory(spellcheckerIndexDir));

+} catch (IOException e) {
+throw new RuntimeException(Cannot open SpellChecker  
index, e);

+}
+}


If this index were updated, what would trigger it to get refreshed by  
this request handler?

Re: svn commit: r519107 - in /lucene/solr/trunk: CHANGES.txt example/exampledocs/spellchecher.xml example/solr/conf/solrconfig.xml lib/lucene-spellchecker-2.2-dev.jar src/java/org/apache/solr/request/

[Otis Gospodnetic]

Hi,
- Original Message 
From: Erik Hatcher [EMAIL PROTECTED]
To: solr-dev@lucene.apache.org
Sent: Friday, March 16, 2007 4:36:25 PM
Subject: Re: svn commit: r519107 - in /lucene/solr/trunk: CHANGES.txt 
example/exampledocs/spellchecher.xml example/solr/conf/solrconfig.xml 
lib/lucene-spellchecker-2.2-dev.jar 
src/java/org/apache/solr/request/SpellCheckerRequestHandler.java


On Mar 16, 2007, at 3:28 PM, [EMAIL PROTECTED] wrote:
 + str name=spellcheckerIndexDir/home/otis/dev/repos/lucene/ 
 solr/trunk/example/solr/data/index/str

And this needs to be changed to something globally relevant.  What  
triggers the creation of this index?   With this part of the example,  
we ought to flesh it out so that it works for anyone trying out Solr,  
right?

OG: Ideally, si.  What should I set it to?  How can I set it to a dir relative 
to solr.home?

 +public void init(NamedList args) {
 +super.init( args );
 +spellcheckerIndexDir = invariants.get 
 (spellcheckerIndexDir);
 +try {
 +spellChecker = new SpellChecker 
 (FSDirectory.getDirectory(spellcheckerIndexDir));
 +} catch (IOException e) {
 +throw new RuntimeException(Cannot open SpellChecker  
 index, e);
 +}
 +}

If this index were updated, what would trigger it to get refreshed by  
this request handler?

OG: ...cmd=rebuild should do it.

Otis

Re: svn commit: r510325 - in /lucene/solr/trunk: CHANGES.txt src/java/org/apache/solr/request/DisMaxRequestHandler.java src/java/org/apache/solr/util/DisMaxParams.java src/test/org/apache/solr/DisMaxR

[Yonik Seeley]

On 2/21/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote:

+ * li ps - (Query Slop) amount of slop on phrase queries explicitly
+ *   specified in the q for qf fields.


Ahh, thanks for the additional code comments... I was a bit confused
since there already was a phrase slop param.

-Yonik

Re: svn commit: r486976 - /incubator/solr/trunk/CHANGES.txt

[Yonik Seeley]

If people really want a separate RELEASE_NOTES.txt, we could copy some
of this stuff out I suppose.  I'm not sure if it's needed at this
point though.

I also figured out that things like CHANGES/RELEASE_NOTES don't need
the ASL too (no one else has it, and  arat didn't complain either).

-Yonik

On 12/14/06, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote:

Author: yonik
Date: Wed Dec 13 21:48:58 2006
New Revision: 486976

URL: http://svn.apache.org/viewvc?view=revrev=486976
Log:
adding intro and release notes type stuff to CHANGES

Modified:
incubator/solr/trunk/CHANGES.txt

Modified: incubator/solr/trunk/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/incubator/solr/trunk/CHANGES.txt?view=diffrev=486976r1=486975r2=486976
==
--- incubator/solr/trunk/CHANGES.txt (original)
+++ incubator/solr/trunk/CHANGES.txt Wed Dec 13 21:48:58 2006
@@ -1,20 +1,12 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements.  See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the License); you may not use this file except in compliance with
-# the License.  You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an AS IS BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
+ Apache Solr Version 1.1.0-dev-incubating
+Release Notes

-= Solr Change Log ==
-$Id$
+Introduction
+
+Apache Solr is an open source enterprise search server based on the Lucene Java
+search library, with XML/HTTP and JSON APIs, hit highlighting, faceted search,
+caching, replication, and a web administration interface. It runs in a Java
+servlet container such as Tomcat.

 Apache Solr is an effort undergoing incubation at The Apache Software
 Foundation (ASF), sponsored by Apache Lucene. Incubation is required of all
@@ -24,6 +16,48 @@
 is not necessarily a reflection of the completeness or stability of the code,
 it does indicate that the project has yet to be fully endorsed by the ASF.

+See http://incubator.apache.org/solr for more information.
+
+
+Getting Started
+---
+You need a Java 1.5 VM or later installed.
+In this release, there is an example Solr server including a bundled
+servlet container in the directory named example.
+See the tutorial at http://incubator.apache.org/solr/tutorial.html
+
+
+$Id$
+
+== Release 1.1.0-dev-incubating, MMDD ==
+
+Status
+--
+This is the first release since Solr joined the Incubator, and brings many
+new features and performance optimizations including highlighting,
+faceted browsing, and JSON/Python/Ruby response formats.
+
+
+Upgrading from previous Solr versions
+-
+Older Apache Solr installations can be upgraded by replacing
+the relevant war file with the new version.  No changes to
+the index format or configuration files are needed.
+
+The default version of the Solr XML response syntax has been changed to 2.2
+This will affect any clients not explicitly specifying the response version.
+
+By default, Solr will no longer use a searcher that has not fully warmed,
+and requests will block in the meantime.  To change back to the previous
+behavior of using a cold searcher in the event there is no other
+warm searcher, see the useColdSearcher config item in solrconfig.xml
+
+The XML response format when adding multiple documents to the collection
+in a single add command has changed to return a single result.
+
+
+Detailed Change List
+

 New Features
  1. added support for setting Lucene's positionIncrementGap

Re: svn commit: r476187 - /incubator/solr/trunk/CHANGES.txt

[Yonik Seeley]

On 11/17/06, Mike Klaas [EMAIL PROTECTED] wrote:

On 11/17/06, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote:
 - 5. Updated to Lucene 2.0 nightly build 2006-09-07
 + 5. Updated to Lucene 2.0 nightly build 2006-09-07, SVN revision 462111

 - 9. Updated to Lucene 2.0 nightly build 2006-11-15
 + 9. Updated to Lucene 2.0 nightly build 2006-11-15, SVN revision 475069

Is there an easy way to determine which revision the nightly build was
cut from?  There is probably something obvious, but since I don't
follow lucene commits, I'm never certain.  Is it sufficient to svn log
around the timestamp of the nightly jar?


In the past, I've known that no commits have happened since the
nightly build, and simply did an svn up and observed the current
revision.

For these historical updates, I just did
svn info -r {2006-09-07T00:03:00Z} http://svn.apache.org/repos/asf

Lucene doesn't have a high commit frequency, but one can always double check via
http://svn.apache.org/viewvc/lucene/java/trunk/

It would be nice to be able to bake the revision into builds somehow though.

-Yonik

Re: svn commit: r476187 - /incubator/solr/trunk/CHANGES.txt

[Chris Hostetter]

: It would be nice to be able to bake the revision into builds somehow though.

you can use svnversion, but there doesn't seem to be an ant plugin for it
- so it would have to be done with an system exec...

http://subversion.tigris.org/faq.html#version-value-in-source
http://www.nabble.com/Inserting-subversion-ID-into-project-source-code-t2634731.html




-Hoss

Re: svn commit: r476187 - /incubator/solr/trunk/CHANGES.txt

[Chris Hostetter]

: And you would want it smart (or a different target) so that it
: wouldn't fail if you weren't building from a subversion dir, or if svn
: wasn't in the path.

sure ... or it could just be part of the dist task.



-Hoss

plugin docs - was: Re: svn commit: r474625 - in /incubator/solr/trunk: CHANGES.txt example/solr/README.txt src/java/org/apache/solr/core/Config.java

[Chris Hostetter]

FYI: I'll update the wiki to include some docs about this a little later
tonight, and then tomorow i'll send out mail to solr-user encouraging
people who currently re-war Solr to include custom code to please try it
out and let us know if there are any problems with it.


-Hoss

Re: Re: svn commit: r471059 - in /incubator/solr/trunk: CHANGES.txt src/java/org/apache/solr/core/SolrCore.java src/test/org/apache/solr/BasicFunctionalityTest.java

[Mike Klaas]

On 11/3/06, Yonik Seeley [EMAIL PROTECTED] wrote:

 : + 9. field boosts weren't being applied and doc boosts were being applied 
to fields (klaas)

 Ouch!  Nice catch.

Heh.  I don't think we've ever used index-time boosts in CNET.


Hopefully there aren't many features that no-one is using *g*.

-Mike

Re: svn commit: r422322 - in /incubator/solr/trunk: CHANGES.txt lib/lucene-core-nightly.jar lib/lucene-highlighter-nightly.jar lib/lucene-snowball-nightly.jar

[Yonik Seeley]

+ 3. Upgrade to Lucene 2.0 nightly build 2006-07-15, lucene SVN revision 422302,


FYI, among other things, this lucene version had the highlighter fixes
needed to work with overlapping tokens (WordDelimiterFilter).

https://issues.apache.org/jira/browse/LUCENE-627

-Yonik

Re: svn commit: r411882 - in /incubator/solr/trunk: CHANGES.txt src/java/org/apache/solr/analysis/KeywordTokenizerFactory.java src/test/org/apache/solr/BasicFunctionalityTest.java src/test/test-files/

[Yonik Seeley]

Ah, thanks... I had been meaning to add that.

-Yonik

On 6/5/06, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote:

Author: hossman
Date: Mon Jun  5 11:20:13 2006
New Revision: 411882

URL: http://svn.apache.org/viewvc?rev=411882view=rev
Log:
adding new KeywordTokenizerFactory

Re: svn commit: r411882 - in /incubator/solr/trunk: CHANGES.txt src/java/org/apache/solr/analysis/KeywordTokenizerFactory.java src/test/org/apache/solr/BasicFunctionalityTest.java src/test/test-files/

[Chris Hostetter]

: Ah, thanks... I had been meaning to add that.

yeah .. the impetus was a coworker who wanted a string field that would
sort in a case insensative way ... i thought about writing a new
SortComparatorSource to do this ... but then figured this would be easier
(and more generally usefull).

I just wish it wasn't neccessary to have all these Factories ... has
anyone done any serious bnchmarking of the cost of reflection in a case
like this? ... if getting a Class by name is the expensive part, we can do
that once when the config is loaded -- it's just a question of how
clazz.newInstance() performs relative to new Foo()



-Hoss

Re: svn commit: r411882 - in /incubator/solr/trunk: CHANGES.txt src/java/org/apache/solr/analysis/KeywordTokenizerFactory.java src/test/org/apache/solr/BasicFunctionalityTest.java src/test/test-files/

[Yonik Seeley]

On 6/5/06, Chris Hostetter [EMAIL PROTECTED] wrote:

I just wish it wasn't neccessary to have all these Factories ... has
anyone done any serious bnchmarking of the cost of reflection in a case
like this? ... if getting a Class by name is the expensive part, we can do
that once when the config is loaded -- it's just a question of how
clazz.newInstance() performs relative to new Foo()


A factory is certainly necessary sometimes.  You don't want to incur
setup time for creating a SynonymMap or a StopSet for every instance
you create.

For simpler tokenizers or token filters, it would be nice to be able
to use them directly.
Perhaps we could look at the class type, and if it's a lucene
Tokenizer or TokenFilter, try instantiating it directly with
newInstance().

At schema creation time, we should probably check if the specified
Tokenizer or TokenFilter has a default constructor.  If it doesn't we
should throw an error right then, not waiting for a confusing runtime
exception.

-Yonik

Re: svn commit: r411882 - in /incubator/solr/trunk: CHANGES.txt src/java/org/apache/solr/analysis/KeywordTokenizerFactory.java src/test/org/apache/solr/BasicFunctionalityTest.java src/test/test-files/

[Chris Hostetter]

: A factory is certainly necessary sometimes.  You don't want to incur
: setup time for creating a SynonymMap or a StopSet for every instance
: you create.

Sure -- i'm not saying we should eliminate the Factoires completely, just
that i wish it wasn't neccessary to write a factory for every
simple Filter/Tokenizer.

: Perhaps we could look at the class type, and if it's a lucene
: Tokenizer or TokenFilter, try instantiating it directly with
: newInstance().

I was thinking of a ReflectionFilterFactory and a
ReflectionTokenizerFactory that take in the class name as an argument --
but if we want to make it automatic that's cool too ... my only concern is
that *if* the reflection performance is noticable, we'd want to make it
explicit to people that they are using it .. so they don't use...

tokenizer class=solr.StandardTokenizer/
..instead of...
tokenizer class=solr.StandardTokenizerFactory/
...just becuase they don't understand the difference, and then complain
that Solr is really slow.

: At schema creation time, we should probably check if the specified
: Tokenizer or TokenFilter has a default constructor.  If it doesn't we
: should throw an error right then, not waiting for a confusing runtime
: exception.

Why would a default constructor matter? isn't what really matters for all
of the Tokenizers and TokenFilters the one arg constructor?



-Hoss