[jira] [Updated] (SOLR-3597) seems like a lot of wasted whitespace at the top of the admin screens

[Stefan Matheis (steffkes) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Matheis (steffkes) updated SOLR-3597:


Component/s: web gui

> seems like a lot of wasted whitespace at the top of the admin screens
> -
>
> Key: SOLR-3597
> URL: https://issues.apache.org/jira/browse/SOLR-3597
> Project: Solr
>  Issue Type: Improvement
>  Components: web gui
>Reporter: Hoss Man
>Assignee: Stefan Matheis (steffkes)
> Fix For: 4.0
>
> Attachments: SOLR-3597-hack.patch
>
>
> because of how the UI is laid out, the solr logo is all by itself above the 
> main table, leaving a lot of whitespace to the right of the solr logo.
> this seems like a waste that should be easily curable?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3597) seems like a lot of wasted whitespace at the top of the admin screens

[Stefan Matheis (steffkes) (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407736#comment-13407736
 ] 

Stefan Matheis (steffkes) commented on SOLR-3597:
-

I really like this change! No idea why the header was left untouched while 
other things also changed .. but this looks really good (at least for me).

bq. .. and i supsect that if we are going to move the logo like this, that para 
tag should move somewhere else ...

played around with the structure, what may fit is: moving the complete header 
block inside the menu-wrapper. with the current stylesheet the environment box 
is moved to the top, but disabling the position-definition in css would help. 
then we would have (see the following as a screen mock):

{code}[logo]
[enviroment]
[navigation]{code}

in that case the navigation will move a bit down, but that should be okay.

bq. most likely this patch shouldn't be used as is w/o other CSS changes or 
what not 

i'll try to have a look at the weekend (beside finishing a few other patches ;o)

> seems like a lot of wasted whitespace at the top of the admin screens
> -
>
> Key: SOLR-3597
> URL: https://issues.apache.org/jira/browse/SOLR-3597
> Project: Solr
>  Issue Type: Improvement
>  Components: web gui
>Reporter: Hoss Man
>Assignee: Stefan Matheis (steffkes)
> Fix For: 4.0
>
> Attachments: SOLR-3597-hack.patch
>
>
> because of how the UI is laid out, the solr logo is all by itself above the 
> main table, leaving a lot of whitespace to the right of the solr logo.
> this seems like a waste that should be easily curable?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3597) seems like a lot of wasted whitespace at the top of the admin screens

[Stefan Matheis (steffkes) (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407731#comment-13407731
 ] 

Stefan Matheis (steffkes) commented on SOLR-3597:
-

bq. (tangent: it's not really clear to me what that "environment" para is for. 
i see javascript that either populates it or kills it depending on what it 
finds in the command line args, but it's not really clear to my why those 
specific args are special ... is this doc'ed anywhere? .. even just an 
HTML/javascript comment explaining what the deal is would be helpful)

this was suggested in the early days of the new UI, here: 
http://www.lucidimagination.com/search/document/5fb42ac7b35091a2 - but you're 
right, it's not clearly documented somewhere else.

> seems like a lot of wasted whitespace at the top of the admin screens
> -
>
> Key: SOLR-3597
> URL: https://issues.apache.org/jira/browse/SOLR-3597
> Project: Solr
>  Issue Type: Improvement
>Reporter: Hoss Man
>Assignee: Stefan Matheis (steffkes)
> Fix For: 4.0
>
> Attachments: SOLR-3597-hack.patch
>
>
> because of how the UI is laid out, the solr logo is all by itself above the 
> main table, leaving a lot of whitespace to the right of the solr logo.
> this seems like a waste that should be easily curable?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3598) Provide option to allow aliased field to be included in query for EDisMax QParser

[Jamie Johnson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jamie Johnson updated SOLR-3598:


Attachment: alias.patch

simple patch which supports this.

> Provide option to allow aliased field to be included in query for EDisMax 
> QParser
> -
>
> Key: SOLR-3598
> URL: https://issues.apache.org/jira/browse/SOLR-3598
> Project: Solr
>  Issue Type: New Feature
>  Components: query parsers
>Affects Versions: 3.6, 4.0, 4.0-ALPHA
>Reporter: Jamie Johnson
>Priority: Minor
> Attachments: alias.patch
>
>
> I currently have a situation where I'd like the original field included in 
> the query, for instance I have several fields with differing granularity, 
> name, firstname and lastname.  Some of my sources differentiate between these 
> so I can fill out firstname and lastname, while others don't and I need to 
> just place this information in the name field.  When querying I'd like to be 
> able to say name:Jamie and have it translated to name:Jamie first_name:Jamie 
> last_name:Jamie.  In order to do this it creates an alias cycle and the 
> EDisMax Query parser throws an exception about it.  Ideally there would be an 
> option to include the original field as part of the query to support this use 
> case.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-3598) Provide option to allow aliased field to be included in query for EDisMax QParser

[Jamie Johnson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407716#comment-13407716
 ] 

Jamie Johnson edited comment on SOLR-3598 at 7/6/12 5:30 AM:
-

simple patch which supports this.  I didn't run any exhaustive tests but the 
aliasing piece works.

  was (Author: jej2003):
simple patch which supports this.
  
> Provide option to allow aliased field to be included in query for EDisMax 
> QParser
> -
>
> Key: SOLR-3598
> URL: https://issues.apache.org/jira/browse/SOLR-3598
> Project: Solr
>  Issue Type: New Feature
>  Components: query parsers
>Affects Versions: 3.6, 4.0, 4.0-ALPHA
>Reporter: Jamie Johnson
>Priority: Minor
> Attachments: alias.patch
>
>
> I currently have a situation where I'd like the original field included in 
> the query, for instance I have several fields with differing granularity, 
> name, firstname and lastname.  Some of my sources differentiate between these 
> so I can fill out firstname and lastname, while others don't and I need to 
> just place this information in the name field.  When querying I'd like to be 
> able to say name:Jamie and have it translated to name:Jamie first_name:Jamie 
> last_name:Jamie.  In order to do this it creates an alias cycle and the 
> EDisMax Query parser throws an exception about it.  Ideally there would be an 
> option to include the original field as part of the query to support this use 
> case.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407710#comment-13407710
 ] 

David Smiley commented on SOLR-2155:


Kristopher,
Perhaps you didn't make the changes to solrconfig.xml that you need to make, 
namely:
{code:xml}


{code}
This is documented on the solr wiki link at the top of this issue.

> Geospatial search using geohash prefixes
> 
>
> Key: SOLR-2155
> URL: https://issues.apache.org/jira/browse/SOLR-2155
> Project: Solr
>  Issue Type: Improvement
>Reporter: David Smiley
>Assignee: David Smiley
> Attachments: GeoHashPrefixFilter.patch, GeoHashPrefixFilter.patch, 
> GeoHashPrefixFilter.patch, 
> SOLR-2155_GeoHashPrefixFilter_with_sorting_no_poly.patch, SOLR.2155.p3.patch, 
> SOLR.2155.p3tests.patch, Solr2155-1.0.2-project.zip, 
> Solr2155-1.0.3-project.zip, Solr2155-1.0.4-project.zip, 
> Solr2155-for-1.0.2-3.x-port.patch
>
>
> {panel:title=NOTICE} The status of this issue is a plugin for Solr 3.x 
> located here: https://github.com/dsmiley/SOLR-2155.  Look at the introductory 
> readme and download the plugin .jar file.  Lucene 4's new spatial module is 
> largely based on this code.  The Solr 4 glue for it should come very soon but 
> as of this writing it's hosted temporarily at https://github.com/spatial4j.  
> For more information on using SOLR-2155 with Solr 3, see 
> http://wiki.apache.org/solr/SpatialSearch#SOLR-2155  This JIRA issue is 
> closed because it won't be committed in its current form.
> {panel}
> There currently isn't a solution in Solr for doing geospatial filtering on 
> documents that have a variable number of points.  This scenario occurs when 
> there is location extraction (i.e. via a "gazateer") occurring on free text.  
> None, one, or many geospatial locations might be extracted from any given 
> document and users want to limit their search results to those occurring in a 
> user-specified area.
> I've implemented this by furthering the GeoHash based work in Lucene/Solr 
> with a geohash prefix based filter.  A geohash refers to a lat-lon box on the 
> earth.  Each successive character added further subdivides the box into a 4x8 
> (or 8x4 depending on the even/odd length of the geohash) grid.  The first 
> step in this scheme is figuring out which geohash grid squares cover the 
> user's search query.  I've added various extra methods to GeoHashUtils (and 
> added tests) to assist in this purpose.  The next step is an actual Lucene 
> Filter, GeoHashPrefixFilter, that uses these geohash prefixes in 
> TermsEnum.seek() to skip to relevant grid squares in the index.  Once a 
> matching geohash grid is found, the points therein are compared against the 
> user's query to see if it matches.  I created an abstraction GeoShape 
> extended by subclasses named PointDistance... and CartesianBox to support 
> different queried shapes so that the filter need not care about these details.
> This work was presented at LuceneRevolution in Boston on October 8th.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-3598) Provide option to allow aliased field to be included in query for EDisMax QParser

[Jamie Johnson (JIRA)]

Jamie Johnson created SOLR-3598:
---

 Summary: Provide option to allow aliased field to be included in 
query for EDisMax QParser
 Key: SOLR-3598
 URL: https://issues.apache.org/jira/browse/SOLR-3598
 Project: Solr
  Issue Type: New Feature
  Components: query parsers
Affects Versions: 3.6, 4.0, 4.0-ALPHA
Reporter: Jamie Johnson
Priority: Minor


I currently have a situation where I'd like the original field included in the 
query, for instance I have several fields with differing granularity, name, 
firstname and lastname.  Some of my sources differentiate between these so I 
can fill out firstname and lastname, while others don't and I need to just 
place this information in the name field.  When querying I'd like to be able to 
say name:Jamie and have it translated to name:Jamie first_name:Jamie 
last_name:Jamie.  In order to do this it creates an alias cycle and the EDisMax 
Query parser throws an exception about it.  Ideally there would be an option to 
include the original field as part of the query to support this use case.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3597) seems like a lot of wasted whitespace at the top of the admin screens

[Hoss Man (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man updated SOLR-3597:
---

Attachment: SOLR-3597-hack.patch

here is a very hackish patch i reverse engineered using the DOM Inspector in 
firefox.

most likely this patch shouldn't be used as is w/o other CSS changes or what 
not -- in particularly i just kind of left the {{ }} paragraph sitting there above the "main" box, and 
i supsect that if we are going to move the logo like this, that para tag should 
move somewhere else ... which would then raise questions about wether the 
"wrapper" div is still needed, which i'm sure affects css, etc...

the point is i'm posting it wo you can see the end result i'm advocating for, 
not because i think this is the right way to get that end result

(tangent: it's not really clear to me what that "environment" para is for.  i 
see javascript that either populates it or kills it depending on what it finds 
in the command line args, but it's not really clear to my why those specific 
args are special ... is this doc'ed anywhere? .. even just an HTML/javascript 
comment explaining  what the deal is would be helpful)

> seems like a lot of wasted whitespace at the top of the admin screens
> -
>
> Key: SOLR-3597
> URL: https://issues.apache.org/jira/browse/SOLR-3597
> Project: Solr
>  Issue Type: Improvement
>Reporter: Hoss Man
>Assignee: Stefan Matheis (steffkes)
> Fix For: 4.0
>
> Attachments: SOLR-3597-hack.patch
>
>
> because of how the UI is laid out, the solr logo is all by itself above the 
> main table, leaving a lot of whitespace to the right of the solr logo.
> this seems like a waste that should be easily curable?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-3597) seems like a lot of wasted whitespace at the top of the admin screens

[Hoss Man (JIRA)]

Hoss Man created SOLR-3597:
--

 Summary: seems like a lot of wasted whitespace at the top of the 
admin screens
 Key: SOLR-3597
 URL: https://issues.apache.org/jira/browse/SOLR-3597
 Project: Solr
  Issue Type: Improvement
Reporter: Hoss Man
Assignee: Stefan Matheis (steffkes)
 Fix For: 4.0


because of how the UI is laid out, the solr logo is all by itself above the 
main table, leaving a lot of whitespace to the right of the solr logo.

this seems like a waste that should be easily curable?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4188) Storing Shapes shouldn't be Strategy dependent

[Chris Male (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407668#comment-13407668
 ] 

Chris Male commented on LUCENE-4188:


{quote}
The client doesn't have to use this method, but in all tests + the Solr 
adapters I don't think there's a reason not to. I found it to be useful, and to 
provide a place to document how it is recommended to store the shape (notice I 
even included the one-liner source in the javadocs). An advantage of it being 
an instance method on the Strategy is that it has convenient access to both the 
field name & SpatialContext. I could make this method final, and I could add 
more documentation that makes it clear that the user is free to store the shape 
in any way they wish since the spatial module doesn't care.
{quote}

I think it is a misnomer to say this is how we recommend a shape is stored.  
That is exactly what I disagree with and why I want it changed from how it is 
currently.  To me the recommended way, if the consumer already has a String 
representation of the Shape that they used to construct the Shape, is to just 
store that String rather than re-parsing the Shape.  I also dont like the use 
of SpatialContext.  It represents Shapes in a non-standard way, sometimes using 
WKT (if you're using JtsSpatialContext) and sometimes not.  Yes it is 
symmetrical with the readShape code, but that is only helpful if you use the 
Context to read your shapes in the first place.

Thinking about it, I think we need to improve the Shape reading code in 
SpatialContext so that it is clear what representation is used and so it is 
possible to extend the functionality when you want to use anything other than 
Strings.

> Storing Shapes shouldn't be Strategy dependent
> --
>
> Key: LUCENE-4188
> URL: https://issues.apache.org/jira/browse/LUCENE-4188
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: modules/spatial
>Reporter: Chris Male
>Assignee: David Smiley
> Attachments: LUCENE-4188_remove_field_storage_from_createField.patch
>
>
> The logic for storing Shape representations seems to be different for each 
> Strategy.  The PrefixTreeStrategy impls store the Shape in WKT, which is nice 
> if you're using WKT but not much help if you're not.  BBoxStrategy doesn't 
> actually store the Shape itself, but a representation of the bounding box.  
> TwoDoubles seems to follow the PrefixTreeStrategy approach, which is 
> surprising since it only indexes Points and they could be stored without 
> using WKT.
> I think we need to consider what storing a Shape means.  If we want to store 
> the Shape itself, then that logic should be standardised and done outside of 
> the Strategys since it is not really related to them.  If we want to store 
> the terms being used by the Strategys to make Shapes queryable, then we need 
> to change the logic in the Strategys to actually do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Loss of JIRA commit links!

[Chris Hostetter]

: It seems as if we've lost JIRA picking up SVN commits recently.
: Anyone know what's going on?

FYI: INFRA was aparently forced to disable this Jira plugin due to some 
performance problems.

Aparently they have an issue open with Atlassian, but there is no stated 
ETA (i can't even find an INFRA issue related to this that folks 
could monitor, but i didn't look very hard)

-Hoss

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3215) We should clone the SolrInputDocument before adding locally and then send that clone to replicas.

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407649#comment-13407649
 ] 

Hoss Man commented on SOLR-3215:


I think this is still pretty damn important:

1) nothing stops people from putting other processors between 
DistributeUpdateProcessorFactory and RunUpdateProcessorFactory
2) several use cases identified above are simpler (or *require*) putting 
processors between DistributeUpdateProcessorFactory and 
RunUpdateProcessorFactory
3) any processor put between DistributeUpdateProcessorFactory and 
RunUpdateProcessorFactory is a time bomb of unpredictability w/o some sort of 
cloning.

> We should clone the SolrInputDocument before adding locally and then send 
> that clone to replicas.
> -
>
> Key: SOLR-3215
> URL: https://issues.apache.org/jira/browse/SOLR-3215
> Project: Solr
>  Issue Type: Improvement
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 4.0
>
> Attachments: SOLR-3215.patch
>
>
> If we don't do this, the behavior is a little unexpected. You cannot avoid 
> having other processors always hit documents twice unless we support using 
> multiple update chains. We have another issue open that should make this 
> better, but I'd like to do this sooner than that. We are going to have to end 
> up cloning anyway when we want to offer the ability to not wait for the local 
> add before sending to replicas.
> Cloning with the current SolrInputDocument, SolrInputField apis is a little 
> scary - there is an Object to contend with - but it seems we can pretty much 
> count on that being a primitive that we don't have to clone?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3215) We should clone the SolrInputDocument before adding locally and then send that clone to replicas.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-3215:
--

Priority: Minor  (was: Major)

we need to reevaluate this in regards to the update processor work that hossman 
did - this may not be needed unless something else prompts it

> We should clone the SolrInputDocument before adding locally and then send 
> that clone to replicas.
> -
>
> Key: SOLR-3215
> URL: https://issues.apache.org/jira/browse/SOLR-3215
> Project: Solr
>  Issue Type: Improvement
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 4.0
>
> Attachments: SOLR-3215.patch
>
>
> If we don't do this, the behavior is a little unexpected. You cannot avoid 
> having other processors always hit documents twice unless we support using 
> multiple update chains. We have another issue open that should make this 
> better, but I'd like to do this sooner than that. We are going to have to end 
> up cloning anyway when we want to offer the ability to not wait for the local 
> add before sending to replicas.
> Cloning with the current SolrInputDocument, SolrInputField apis is a little 
> scary - there is an Object to contend with - but it seems we can pretty much 
> count on that being a primitive that we don't have to clone?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3563) Collection in ZK not deleted when all shards has been unloaded

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407646#comment-13407646
 ] 

Mark Miller commented on SOLR-3563:
---

This was added as part of SOLR-3488 - I'll make a changes entry.

> Collection in ZK not deleted when all shards has been unloaded
> --
>
> Key: SOLR-3563
> URL: https://issues.apache.org/jira/browse/SOLR-3563
> Project: Solr
>  Issue Type: Bug
>  Components: multicore, SolrCloud
>Affects Versions: 4.0
> Environment: Same as SOLR-3561
>Reporter: Per Steffensen
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 4.0
>
>
> Same scanario as SOLR-3561 - deleting shards/cores using CoreAdmin/UNLOAD 
> command.
> I have noticed that when I have done CoreAdmin/UNLOAD for all shard under a 
> collection, that the collection and all its slices are still present in ZK 
> under /collections. I might be ok since the operation is called UNLOAD, but I 
> basically want to delete an entire collection and all data related to it 
> (including information about it in ZK).
> A delete-collection operation, that also deletes info about the collection 
> under /collections in ZK, would be very nice! Or a delete-shard/core 
> operation and then some nice logic that detects when all shards belonging to 
> a collection has been deleted, and when that has happened deletes info about 
> the collection under /collections in ZK.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3562) Data folder not deleted during unload

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407645#comment-13407645
 ] 

Mark Miller commented on SOLR-3562:
---

This was completed as part of SOLR-3488 - I'll add a changes entry.

> Data folder not deleted during unload
> -
>
> Key: SOLR-3562
> URL: https://issues.apache.org/jira/browse/SOLR-3562
> Project: Solr
>  Issue Type: Bug
>  Components: multicore, SolrCloud
>Affects Versions: 4.0
> Environment: Same as SOLR-3561
>Reporter: Per Steffensen
>Assignee: Mark Miller
>Priority: Minor
>
> Same scanario as SOLR-3561 - deleting shards/cores using CoreAdmin/UNLOAD 
> command.
> I have noticed that when doing CoreAdmin/UNLOAD, the data-folder on disk 
> belonging to the shard/core that has been unloaded is not deleted. I might be 
> ok since the operation is called UNLOAD, but I basically want to delete a 
> shard/core and all data related to it (including its data-folder).
> Dont we have a delete shard/core operation? Or what do I need to do? Do I 
> have to manually delete the data-folder myself after having unloaded?
> A delete-shard/core or even a delete-collection operation would be very nice!

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2633) Make SolrDispatchFilter testable and add tests

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407644#comment-13407644
 ] 

Mark Miller commented on SOLR-2633:
---

Ping?

> Make SolrDispatchFilter testable and add tests
> --
>
> Key: SOLR-2633
> URL: https://issues.apache.org/jira/browse/SOLR-2633
> Project: Solr
>  Issue Type: Improvement
>  Components: search
>Affects Versions: 3.1, 3.2, 3.3
>Reporter: Edoardo Tosca
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 4.0
>
> Attachments: SOLR-2633-tests-only.patch, SOLR-2633-tests-only.patch
>
>
> I have ideas for possible extensions/enhancements to the SolrDispatchFilter. 
> However, as it doesn't have any tests, making safe enhancements is difficult. 
> Given its monolithic nature, it is hard to test. Therefore, I am proposing to 
> refactor it to make it testable, and to provide tests for it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3093) Decide destiny of LuceneQueryOptimizer (solrconfig.xml: )

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407642#comment-13407642
 ] 

Mark Miller commented on SOLR-3093:
---

So I commented out the below. I would assume it was because Hits was removed. I 
don't recall if anything was actually using that method - perhaps not - if so, 
I'm not sure what I switched those callers to use - or if the optimizer could 
be used there if it made sense - I'll have to dig further when I get chance:

{code}
  public Hits search(Query query, Filter filter, Sort sort) throws 
IOException {
// todo - when Solr starts accepting filters, need to
// change this conditional check (filter!=null) and create a new 
filter
// that ANDs them together if it already exists.

if (optimizer==null || filter!=null || !(query instanceof 
BooleanQuery)
) {
  return super.search(query,filter,sort);
} else {
  Query[] newQuery = new Query[1];
  Filter[] newFilter = new Filter[1];
  optimizer.optimize((BooleanQuery)query, this, 0, newQuery, 
newFilter);

  return super.search(newQuery[0], newFilter[0], sort);
}
  } 
{code}

> Decide destiny of LuceneQueryOptimizer (solrconfig.xml: 
> )
> 
>
> Key: SOLR-3093
> URL: https://issues.apache.org/jira/browse/SOLR-3093
> Project: Solr
>  Issue Type: Task
>Reporter: Jan Høydahl
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 4.0
>
>
> SolrConfig.java still tries to parse 
> But the only user of this param was SolrIndexSearcher.java line 366-381 which 
> is commented out since 3.1.
> From Solr3.6 we print a warning if we find this option is found in 
> solrconfig.xml (SOLR-1052):
> {noformat}
> WARN: solrconfig.xml:  is currently not implemented 
> and has no effect.
> {noformat}
> However, the dead, commented code is still in place in {{SolrConfig.java, 
> SolrIndexSearcher.Java}}. We should decide what do do, either re-implement 
> boolTofilterOptimizer or remove it completely.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3353) Commented examples in solrconfig.xml don't account for SolrCloud in URP chains

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-3353:
--

Priority: Minor  (was: Major)

> Commented examples in solrconfig.xml don't account for SolrCloud in URP chains
> --
>
> Key: SOLR-3353
> URL: https://issues.apache.org/jira/browse/SOLR-3353
> Project: Solr
>  Issue Type: Bug
>  Components: Schema and Analysis
>Affects Versions: 4.0
>Reporter: Benson Margulies
>Assignee: Mark Miller
>Priority: Minor
>
> The example solrconfig.xml includes several items like:
> {code}
>  
> class="org.apache.solr.update.processor.TikaLanguageIdentifierUpdateProcessorFactory">
>  text,title,subject,description
>  language_s
>  en
>
>
>
>  
> {code}
> that leave out the DistributedUpdateProcessorFactory.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3353) Commented examples in solrconfig.xml don't account for SolrCloud in URP chains

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407638#comment-13407638
 ] 

Mark Miller commented on SOLR-3353:
---

This should be fine now - the distrib processor is auto injected if it's not in 
the chain. We could still add it explicitly if people felt we should though.

> Commented examples in solrconfig.xml don't account for SolrCloud in URP chains
> --
>
> Key: SOLR-3353
> URL: https://issues.apache.org/jira/browse/SOLR-3353
> Project: Solr
>  Issue Type: Bug
>  Components: Schema and Analysis
>Affects Versions: 4.0
>Reporter: Benson Margulies
>Assignee: Mark Miller
>
> The example solrconfig.xml includes several items like:
> {code}
>  
> class="org.apache.solr.update.processor.TikaLanguageIdentifierUpdateProcessorFactory">
>  text,title,subject,description
>  language_s
>  en
>
>
>
>  
> {code}
> that leave out the DistributedUpdateProcessorFactory.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

[Kristopher Davidsohn (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407625#comment-13407625
 ] 

Kristopher Davidsohn commented on SOLR-2155:


Hi, I have the latest version of this patch, and in this version (and prior 
ones I've used as well actually) I could not get the distance sorting to work. 
I have a geohash field set up like so in the schema: 
{code:xml} 
{code}

and a sample query I am testing: 
{code}q=(locName%3Ario^5.0)&sort=geodist()+asc&rows=15&start=0&fq={!bbox}&sfield=locLatLon_hash&pt=40.2114%2C-111.6980&d=80.45&qt=standard{code}
 

The results come out in no real discernable order (not distance, and not score) 
and it seems like I may be missing something. Does anyone have any advice or 
ideas on what might be the issue? Or is distance sorting not supported in this 
particular case?

> Geospatial search using geohash prefixes
> 
>
> Key: SOLR-2155
> URL: https://issues.apache.org/jira/browse/SOLR-2155
> Project: Solr
>  Issue Type: Improvement
>Reporter: David Smiley
>Assignee: David Smiley
> Attachments: GeoHashPrefixFilter.patch, GeoHashPrefixFilter.patch, 
> GeoHashPrefixFilter.patch, 
> SOLR-2155_GeoHashPrefixFilter_with_sorting_no_poly.patch, SOLR.2155.p3.patch, 
> SOLR.2155.p3tests.patch, Solr2155-1.0.2-project.zip, 
> Solr2155-1.0.3-project.zip, Solr2155-1.0.4-project.zip, 
> Solr2155-for-1.0.2-3.x-port.patch
>
>
> {panel:title=NOTICE} The status of this issue is a plugin for Solr 3.x 
> located here: https://github.com/dsmiley/SOLR-2155.  Look at the introductory 
> readme and download the plugin .jar file.  Lucene 4's new spatial module is 
> largely based on this code.  The Solr 4 glue for it should come very soon but 
> as of this writing it's hosted temporarily at https://github.com/spatial4j.  
> For more information on using SOLR-2155 with Solr 3, see 
> http://wiki.apache.org/solr/SpatialSearch#SOLR-2155  This JIRA issue is 
> closed because it won't be committed in its current form.
> {panel}
> There currently isn't a solution in Solr for doing geospatial filtering on 
> documents that have a variable number of points.  This scenario occurs when 
> there is location extraction (i.e. via a "gazateer") occurring on free text.  
> None, one, or many geospatial locations might be extracted from any given 
> document and users want to limit their search results to those occurring in a 
> user-specified area.
> I've implemented this by furthering the GeoHash based work in Lucene/Solr 
> with a geohash prefix based filter.  A geohash refers to a lat-lon box on the 
> earth.  Each successive character added further subdivides the box into a 4x8 
> (or 8x4 depending on the even/odd length of the geohash) grid.  The first 
> step in this scheme is figuring out which geohash grid squares cover the 
> user's search query.  I've added various extra methods to GeoHashUtils (and 
> added tests) to assist in this purpose.  The next step is an actual Lucene 
> Filter, GeoHashPrefixFilter, that uses these geohash prefixes in 
> TermsEnum.seek() to skip to relevant grid squares in the index.  Once a 
> matching geohash grid is found, the points therein are compared against the 
> user's query to see if it matches.  I created an abstraction GeoShape 
> extended by subclasses named PointDistance... and CartesianBox to support 
> different queried shapes so that the filter need not care about these details.
> This work was presented at LuceneRevolution in Boston on October 8th.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2225) CoreContainer#register should use checkDefault to normalize the core name

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407613#comment-13407613
 ] 

Mark Miller commented on SOLR-2225:
---

I've updated this test and somehow this passes now - another issue must have 
solved it. I'll just commit the new test.

> CoreContainer#register should use checkDefault to normalize the core name
> -
>
> Key: SOLR-2225
> URL: https://issues.apache.org/jira/browse/SOLR-2225
> Project: Solr
>  Issue Type: Bug
>  Components: multicore
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 4.0, 5.0
>
> Attachments: SOLR-2225.patch
>
>
> fail case:
> start with default collection set to collection1
> remove core collection1
> default collection on CoreContainer is still set to collection1
> add core collection1
> it doesn't act like the default core
> we might do as the summary suggests, or when the default core is removed, we 
> reset to no default core until one is again explicitly set

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-2225) CoreContainer#register should use checkDefault to normalize the core name

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-2225:
--

Fix Version/s: 5.0

> CoreContainer#register should use checkDefault to normalize the core name
> -
>
> Key: SOLR-2225
> URL: https://issues.apache.org/jira/browse/SOLR-2225
> Project: Solr
>  Issue Type: Bug
>  Components: multicore
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 4.0, 5.0
>
> Attachments: SOLR-2225.patch
>
>
> fail case:
> start with default collection set to collection1
> remove core collection1
> default collection on CoreContainer is still set to collection1
> add core collection1
> it doesn't act like the default core
> we might do as the summary suggests, or when the default core is removed, we 
> reset to no default core until one is again explicitly set

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-2225) CoreContainer#register should use checkDefault to normalize the core name

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller resolved SOLR-2225.
---

Resolution: Fixed

> CoreContainer#register should use checkDefault to normalize the core name
> -
>
> Key: SOLR-2225
> URL: https://issues.apache.org/jira/browse/SOLR-2225
> Project: Solr
>  Issue Type: Bug
>  Components: multicore
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 4.0, 5.0
>
> Attachments: SOLR-2225.patch
>
>
> fail case:
> start with default collection set to collection1
> remove core collection1
> default collection on CoreContainer is still set to collection1
> add core collection1
> it doesn't act like the default core
> we might do as the summary suggests, or when the default core is removed, we 
> reset to no default core until one is again explicitly set

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-141) Errors/Exceptions should be formated by ResponseWriter

[Greg Bowyer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Greg Bowyer updated SOLR-141:
-

Attachment: SOLR-141-3.x-Backport.patch

Backport of SOLR-141 to 3.x 

> Errors/Exceptions should be formated by ResponseWriter
> --
>
> Key: SOLR-141
> URL: https://issues.apache.org/jira/browse/SOLR-141
> Project: Solr
>  Issue Type: Wish
>Reporter: Hoss Man
>Assignee: Ryan McKinley
> Fix For: 4.0
>
> Attachments: SOLR-141-3.x-Backport.patch, SOLR-141-sendError.patch, 
> SOLR-141-sendError.patch, error-response.patch, 
> solr-exception-writer-solr-1.2.diff, solr-exception-writer-v2.diff, 
> solr-exception-writer-v3.diff
>
>
> Whenever possible, the Solr Dispatcher should to let the ResponseWriter 
> format Exceptions using the format the user expects -- this should in no way 
> change the fact that Exceptions currently generate non 200 HTTP status codes, 
> nor should it prevent the Dispatcher from using the exception message as the 
> HTTP status message -- but clients that want the full details of the error 
> should be able to parse them in the format they expected based on the request.
> this would also give RequestHandlers the oportunity to return "partial" 
> results - by adding both whatever results they have to the Response as well 
> as the Exception they encountered.
> situations where this can't happen are obviously:
>   * Exception thrown by ResponseWriter
>   * Exception thrown so early in the request thta the DIspather doesn't know 
> which ResposneWriter the client wanted.
> ...in those cases, plain text is a wise choice.
> thing that would probably need to be done to make this work:
> 1) if the Dispatcher catches an exception, it should call 
> SolrQueryResponse.setException, set the HTTP status code/message as it 
> currently does, but then hand off to the ResponseWriter.
> 2) Dispatcher needs to check SolrQueryResponse.getException and set the HTTP 
> status code/message just like if it caught the exception itself.
> 3) all of the ResponseWriters should start looking at 
> SolrQueryResponse.getException if they aren't already, and formatting it in a 
> usefull way.
> 4) if the ResponseWriter throws an exception, Dispatcher needs to return a 
> nice plain text error page
> extension to this idea... add a new method to ResponseWriter to generate a 
> generic error message in the appropriate format that Dispatcher can use if 
> the ResponseWriter throws an exception (as a backup before resorting to plain 
> text)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Closed] (SOLR-3596) Backport of SOLR-141 to 3.x

[Greg Bowyer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Greg Bowyer closed SOLR-3596.
-

Resolution: Invalid

Patch moved to the original issue SOLR-141

> Backport of SOLR-141 to 3.x
> ---
>
> Key: SOLR-3596
> URL: https://issues.apache.org/jira/browse/SOLR-3596
> Project: Solr
>  Issue Type: Improvement
>Reporter: Greg Bowyer
> Attachments: 
> SOLR-3569-Backport-of-SOLR-141-Better-exception-handling.patch
>
>
> This is an attempt at a backport of the functionality introduced with 
> SOLR-141 I don't necessarily expect many more 3.x releases but it at least 
> makes error reporting a bit better on 3.x

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3596) Backport of SOLR-141 to 3.x

[Greg Bowyer (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407595#comment-13407595
 ] 

Greg Bowyer commented on SOLR-3596:
---

Good idea, I will do that

> Backport of SOLR-141 to 3.x
> ---
>
> Key: SOLR-3596
> URL: https://issues.apache.org/jira/browse/SOLR-3596
> Project: Solr
>  Issue Type: Improvement
>Reporter: Greg Bowyer
> Attachments: 
> SOLR-3569-Backport-of-SOLR-141-Better-exception-handling.patch
>
>
> This is an attempt at a backport of the functionality introduced with 
> SOLR-141 I don't necessarily expect many more 3.x releases but it at least 
> makes error reporting a bit better on 3.x

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3596) Backport of SOLR-141 to 3.x

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407594#comment-13407594
 ] 

Hoss Man commented on SOLR-3596:


bq. I have a need for this fix and as such I only put it on jira should others 
need to apply this patch. 

Ah ok ... in that case i would just put it as (clearly named) an attachment in 
SOLR-141 so it's easy to find if people in a similar situation go looking for 
it.

> Backport of SOLR-141 to 3.x
> ---
>
> Key: SOLR-3596
> URL: https://issues.apache.org/jira/browse/SOLR-3596
> Project: Solr
>  Issue Type: Improvement
>Reporter: Greg Bowyer
> Attachments: 
> SOLR-3569-Backport-of-SOLR-141-Better-exception-handling.patch
>
>
> This is an attempt at a backport of the functionality introduced with 
> SOLR-141 I don't necessarily expect many more 3.x releases but it at least 
> makes error reporting a bit better on 3.x

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4190) IndexWriter deletes non-Lucene files

[Robert Muir (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Muir updated LUCENE-4190:


Attachment: LUCENE-4190.patch

Updated patch with the regex instead of the underscore.

additionally, dont fire off IndexFileDeleter's nuking of files if there is no 
current segments file. 

This means if you screw up and pick c:\, we do nothing. if you go and commit 
documents to c:\, then the next time you fire up indexwriter it will do the 
nuking as its then actually an index directory. 

But this gives you an additional chance to catch your mistake besides just the 
regex: plus its just a null check/optimization if you want to look at it.


> IndexWriter deletes non-Lucene files
> 
>
> Key: LUCENE-4190
> URL: https://issues.apache.org/jira/browse/LUCENE-4190
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Robert Muir
> Fix For: 4.0, 5.0
>
> Attachments: LUCENE-4190.patch, LUCENE-4190.patch, LUCENE-4190.patch, 
> LUCENE-4190.patch
>
>
> Carl Austin raised a good issue in a comment on my Lucene 4.0.0 alpha blog 
> post: 
> http://blog.mikemccandless.com/2012/07/lucene-400-alpha-at-long-last.html
> IndexWriter will now (as of 4.0) delete all foreign files from the index 
> directory.  We made this change because Codecs are free to write to any files 
> now, so the space of filenames is hard to "bound".
> But if the user accidentally uses the wrong directory (eg c:/) then we will 
> in fact delete important stuff.
> I think we can at least use some simple criteria (must start with _, maybe 
> must fit certain pattern eg _(_X).Y), so we are much less likely to 
> delete a non-Lucene file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3596) Backport of SOLR-141 to 3.x

[Greg Bowyer (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407588#comment-13407588
 ] 

Greg Bowyer commented on SOLR-3596:
---

I somewhat agree, I dont think this should be included in 3.6.1 (if/as that 
appears), I have a need for this fix and as such I only put it on jira should 
others need to apply this patch. Maybe if there is a point release to 3.7 I 
would sing a slightly different tune, but I dont expect a 3.7.

> Backport of SOLR-141 to 3.x
> ---
>
> Key: SOLR-3596
> URL: https://issues.apache.org/jira/browse/SOLR-3596
> Project: Solr
>  Issue Type: Improvement
>Reporter: Greg Bowyer
> Attachments: 
> SOLR-3569-Backport-of-SOLR-141-Better-exception-handling.patch
>
>
> This is an attempt at a backport of the functionality introduced with 
> SOLR-141 I don't necessarily expect many more 3.x releases but it at least 
> makes error reporting a bit better on 3.x

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3596) Backport of SOLR-141 to 3.x

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407585#comment-13407585
 ] 

Hoss Man commented on SOLR-3596:



if/when 3.6.1 comes out it should only have bug fixes, and i don't think 
SOLR-141 can be justifiably called a bug fix.

In particular i'm worried about people who have applications that are relying 
on the eccentricities of pre SOLR-141 HTTP response, and that upgrading from 
3.6->3.6.1 might not be trivial for them, which would either mean sacraficing 
other 3.6.1 fixes, or changing their own code.

> Backport of SOLR-141 to 3.x
> ---
>
> Key: SOLR-3596
> URL: https://issues.apache.org/jira/browse/SOLR-3596
> Project: Solr
>  Issue Type: Improvement
>Reporter: Greg Bowyer
> Attachments: 
> SOLR-3569-Backport-of-SOLR-141-Better-exception-handling.patch
>
>
> This is an attempt at a backport of the functionality introduced with 
> SOLR-141 I don't necessarily expect many more 3.x releases but it at least 
> makes error reporting a bit better on 3.x

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Closed] (SOLR-2871) Add the ability to assert SOLR cache contents

[Greg Bowyer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-2871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Greg Bowyer closed SOLR-2871.
-

Resolution: Invalid

When I raised this issue I had a poor understanding about what solr caches 
contain

> Add the ability to assert SOLR cache contents
> -
>
> Key: SOLR-2871
> URL: https://issues.apache.org/jira/browse/SOLR-2871
> Project: Solr
>  Issue Type: Improvement
>  Components: search
>Affects Versions: 3.3
>Reporter: Greg Bowyer
>  Labels: cache, patch
> Attachments: 0001-Added-in-assertions-for-solr-caches.patch
>
>
> Due to investigations on my part in looking into a possible misbehaviour (or
> misunderstanding of the correct behaviour) of the filter cache this commit
> adds in the ability to assert that the form of items presented to solr caches
> are valid and expected.
> The interface for this is known as a CacheGuarantor, by default this interface
> resolves a NoOp implementation that does nothing (and should be nicely
> eliminated by the JIT as dead code).
> For the filterCache a basic (and potentially incorrect) guarantor is setup
> that demands all keys presented to the filter cache are FilterQueries.
> This currently makes function queries fail unit tests, which may (or may
> not !) be a bug (and I am guessing probably not here since I am clearly 
> missing some deeper truth)
> I tend to work off github so the patch is also on the tip of this branch 
> https://github.com/GregBowyer/lucene-solr/commits/cache-guarantors

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3596) Backport of SOLR-141 to 3.x

[Greg Bowyer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Greg Bowyer updated SOLR-3596:
--

Attachment: SOLR-3569-Backport-of-SOLR-141-Better-exception-handling.patch

> Backport of SOLR-141 to 3.x
> ---
>
> Key: SOLR-3596
> URL: https://issues.apache.org/jira/browse/SOLR-3596
> Project: Solr
>  Issue Type: Improvement
>Reporter: Greg Bowyer
> Attachments: 
> SOLR-3569-Backport-of-SOLR-141-Better-exception-handling.patch
>
>
> This is an attempt at a backport of the functionality introduced with 
> SOLR-141 I don't necessarily expect many more 3.x releases but it at least 
> makes error reporting a bit better on 3.x

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-2702) Add support for NRTCachingDirectory

[Yonik Seeley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yonik Seeley updated SOLR-2702:
---

Attachment: SOLR-2702.patch

Quick patch for the issue of getting the index directory.  No new tests, but 
all existing tests pass.

> Add support for NRTCachingDirectory
> ---
>
> Key: SOLR-2702
> URL: https://issues.apache.org/jira/browse/SOLR-2702
> Project: Solr
>  Issue Type: Improvement
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 4.0, 5.0
>
> Attachments: SOLR-2702.patch
>
>
> would be nice to have this option for the new NRT support

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-3596) Backport of SOLR-141 to 3.x

[Greg Bowyer (JIRA)]

Greg Bowyer created SOLR-3596:
-

 Summary: Backport of SOLR-141 to 3.x
 Key: SOLR-3596
 URL: https://issues.apache.org/jira/browse/SOLR-3596
 Project: Solr
  Issue Type: Improvement
Reporter: Greg Bowyer


This is an attempt at a backport of the functionality introduced with SOLR-141 
I don't necessarily expect many more 3.x releases but it at least makes error 
reporting a bit better on 3.x



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4190) IndexWriter deletes non-Lucene files

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407562#comment-13407562
 ] 

Robert Muir commented on LUCENE-4190:
-

I think the regex matching the current behavior (as restrictive as i can get) 
is:

{noformat}
_(_.*)?\..*
{noformat}


> IndexWriter deletes non-Lucene files
> 
>
> Key: LUCENE-4190
> URL: https://issues.apache.org/jira/browse/LUCENE-4190
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Robert Muir
> Fix For: 4.0, 5.0
>
> Attachments: LUCENE-4190.patch, LUCENE-4190.patch, LUCENE-4190.patch
>
>
> Carl Austin raised a good issue in a comment on my Lucene 4.0.0 alpha blog 
> post: 
> http://blog.mikemccandless.com/2012/07/lucene-400-alpha-at-long-last.html
> IndexWriter will now (as of 4.0) delete all foreign files from the index 
> directory.  We made this change because Codecs are free to write to any files 
> now, so the space of filenames is hard to "bound".
> But if the user accidentally uses the wrong directory (eg c:/) then we will 
> in fact delete important stuff.
> I think we can at least use some simple criteria (must start with _, maybe 
> must fit certain pattern eg _(_X).Y), so we are much less likely to 
> delete a non-Lucene file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-4.x-Linux-Java6-64 - Build # 394 - Failure!

[Policeman Jenkins Server]

Build: 
http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux-Java6-64/394/

All tests passed

Build Log:
[...truncated 18143 lines...]
javadocs-lint:

[...truncated 609 lines...]
javadocs-lint:

[...truncated 10 lines...]
Looks like the node went offline during the build. Check the slave log for the 
details.FATAL: /var/lib/jenkins/slave-null.log (No such file or directory)
java.io.FileNotFoundException: /var/lib/jenkins/slave-null.log (No such file or 
directory)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.(RandomAccessFile.java:216)
at 
org.kohsuke.stapler.framework.io.LargeText$FileSession.(LargeText.java:351)
at org.kohsuke.stapler.framework.io.LargeText$1.open(LargeText.java:75)
at 
org.kohsuke.stapler.framework.io.LargeText.writeLogTo(LargeText.java:164)
at 
hudson.console.AnnotatedLargeText.writeHtmlTo(AnnotatedLargeText.java:158)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:512)
at hudson.model.Run.execute(Run.java:1488)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:236)




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4190) IndexWriter deletes non-Lucene files

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407550#comment-13407550
 ] 

Michael McCandless commented on LUCENE-4190:


I think {noformat}_.*{noformat} or {noformat}__*.{noformat} is better 
than simply _*?

Ie, it further reduces the risk of deleting a non-Lucene file, while
not restricting the codec at all (this is already its private
namespace)?  No codec should be writing files outside of this space
(and if somehow one needs to in the future, we can cross that bridge
at that point?).

It's of course still not perfect but I think it's as close as we should try
to get.


> IndexWriter deletes non-Lucene files
> 
>
> Key: LUCENE-4190
> URL: https://issues.apache.org/jira/browse/LUCENE-4190
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Robert Muir
> Fix For: 4.0, 5.0
>
> Attachments: LUCENE-4190.patch, LUCENE-4190.patch, LUCENE-4190.patch
>
>
> Carl Austin raised a good issue in a comment on my Lucene 4.0.0 alpha blog 
> post: 
> http://blog.mikemccandless.com/2012/07/lucene-400-alpha-at-long-last.html
> IndexWriter will now (as of 4.0) delete all foreign files from the index 
> directory.  We made this change because Codecs are free to write to any files 
> now, so the space of filenames is hard to "bound".
> But if the user accidentally uses the wrong directory (eg c:/) then we will 
> in fact delete important stuff.
> I think we can at least use some simple criteria (must start with _, maybe 
> must fit certain pattern eg _(_X).Y), so we are much less likely to 
> delete a non-Lucene file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3595) Currency types do not support range queries when multiValued

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407547#comment-13407547
 ] 

Uwe Schindler commented on SOLR-3595:
-

Multivalued does not work with fields that consist of more than one Lucene 
field behind, because the field vales cannot be corelated. We should disallow 
multi value and throw exception. This is similar to LatLong fields I think.

> Currency types do not support range queries when multiValued
> 
>
> Key: SOLR-3595
> URL: https://issues.apache.org/jira/browse/SOLR-3595
> Project: Solr
>  Issue Type: Bug
>  Components: Schema and Analysis
>Affects Versions: 4.0-ALPHA
>Reporter: Erick Erickson
>Priority: Minor
>
> You can define the currency type as multiValued. However, if you do (and have 
> more than one value), range queries, at least, do not work. See the thread 
> titled "Filtering a query by range returning unexpected results".
> I'm not at all sure that currency type _should_ support multivalued. For 
> instance, how would one handle storing multiple values for a currency type in 
> different currencies (e.g. USD and EUR)? I don't know enough about the 
> internals to understand if it's possible, this JIRA is the result of a 
> question on the users list.
> If we decide that currency should _not_ support multiValued, it seems a check 
> at startup is in order on the "fail early, fail loudly" principle.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-3595) Currency types do not support range queries when multiValued

[Erick Erickson (JIRA)]

Erick Erickson created SOLR-3595:


 Summary: Currency types do not support range queries when 
multiValued
 Key: SOLR-3595
 URL: https://issues.apache.org/jira/browse/SOLR-3595
 Project: Solr
  Issue Type: Bug
  Components: Schema and Analysis
Affects Versions: 4.0-ALPHA
Reporter: Erick Erickson
Priority: Minor


You can define the currency type as multiValued. However, if you do (and have 
more than one value), range queries, at least, do not work. See the thread 
titled "Filtering a query by range returning unexpected results".

I'm not at all sure that currency type _should_ support multivalued. For 
instance, how would one handle storing multiple values for a currency type in 
different currencies (e.g. USD and EUR)? I don't know enough about the 
internals to understand if it's possible, this JIRA is the result of a question 
on the users list.

If we decide that currency should _not_ support multiValued, it seems a check 
at startup is in order on the "fail early, fail loudly" principle.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-3252) QueryElevationComponent has only partial support for SolrCloud and does not try to read the elevation file from zookeeper.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller resolved SOLR-3252.
---

Resolution: Fixed

this was fixed as part of SOLR-2949

> QueryElevationComponent has only partial support for SolrCloud and does not 
> try to read the elevation file from zookeeper.
> --
>
> Key: SOLR-3252
> URL: https://issues.apache.org/jira/browse/SOLR-3252
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 4.0
>
>
> It already has a quick work around for checking if the elevate file exists, 
> and it uses the resource loader, but because it gets the inputstream from the 
> filesystem and passes it explicitly, the ZooKeeper SolrResourceLoader doesn't 
> get to do it's job.
> To be very helpful this issue also relies on SOLR-2949: 
> QueryElevationComponent does not fully support distributed search

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2702) Add support for NRTCachingDirectory

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407540#comment-13407540
 ] 

Yonik Seeley commented on SOLR-2702:


Easiest approach for now would be to add a NRTCachingDirectory.getDelegate() 
call and then also check for that.

> Add support for NRTCachingDirectory
> ---
>
> Key: SOLR-2702
> URL: https://issues.apache.org/jira/browse/SOLR-2702
> Project: Solr
>  Issue Type: Improvement
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 4.0, 5.0
>
>
> would be nice to have this option for the new NRT support

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3460) Improve cmd line config bootstrap tool.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-3460:
--

Priority: Major  (was: Minor)

Still need to add to the wiki.

> Improve cmd line config bootstrap tool.
> ---
>
> Key: SOLR-3460
> URL: https://issues.apache.org/jira/browse/SOLR-3460
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 4.0
>
> Attachments: SOLR-3460.patch
>
>
> Improve cmd line tool for bootstrapping config sets. Rather than take a 
> config set name and directory, make it work like -Dboostrap_conf=true and 
> read solr.xml to find config sets. Config sets will be named after the 
> collection and auto linked to the identically named collection.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-2702) Add support for NRTCachingDirectory

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-2702:
--

Fix Version/s: 5.0

> Add support for NRTCachingDirectory
> ---
>
> Key: SOLR-2702
> URL: https://issues.apache.org/jira/browse/SOLR-2702
> Project: Solr
>  Issue Type: Improvement
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 4.0, 5.0
>
>
> would be nice to have this option for the new NRT support

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2702) Add support for NRTCachingDirectory

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407531#comment-13407531
 ] 

Mark Miller commented on SOLR-2702:
---

Another issue currently is that this will not work with SolrCloud correctly 
because it is not detected as an FSDir it appears, and so we don't get the 
index dir, and replication does not work.

> Add support for NRTCachingDirectory
> ---
>
> Key: SOLR-2702
> URL: https://issues.apache.org/jira/browse/SOLR-2702
> Project: Solr
>  Issue Type: Improvement
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 4.0, 5.0
>
>
> would be nice to have this option for the new NRT support

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-1770) move default example core config/data into a collection1 folder

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407530#comment-13407530
 ] 

Mark Miller commented on SOLR-1770:
---

I need to remember to check the Solr tutorial around this ... we really need to 
start doing versioned documentation...

Our confluence wikis were just created the other day I read...

> move default example core config/data into a collection1 folder
> ---
>
> Key: SOLR-1770
> URL: https://issues.apache.org/jira/browse/SOLR-1770
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: 1.4
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Critical
> Fix For: 4.0, 5.0
>
> Attachments: SOLR-1770.patch
>
>
> This is a better starting point for adding more cores - perhaps we can also 
> get rid of multi-core example

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3312) Break out StorableField from IndexableField

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407524#comment-13407524
 ] 

Uwe Schindler commented on LUCENE-3312:
---

And as noted before: In Lucene core we dont use extrenal dependencies, so we 
have a compile failure because of AbstractIterator from Google Collect, we have 
to put this one into Lucene utils.

> Break out StorableField from IndexableField
> ---
>
> Key: LUCENE-3312
> URL: https://issues.apache.org/jira/browse/LUCENE-3312
> Project: Lucene - Java
>  Issue Type: Improvement
>  Components: core/index
>Reporter: Michael McCandless
>Assignee: Nikola Tankovic
>  Labels: gsoc2012, lucene-gsoc-12
> Fix For: Field Type branch
>
> Attachments: lucene-3312-patch-01.patch, lucene-3312-patch-02.patch, 
> lucene-3312-patch-03.patch, lucene-3312-patch-04.patch, 
> lucene-3312-patch-05.patch, lucene-3312-patch-06.patch
>
>
> In the field type branch we have strongly decoupled
> Document/Field/FieldType impl from the indexer, by having only a
> narrow API (IndexableField) passed to IndexWriter.  This frees apps up
> use their own "documents" instead of the "user-space" impls we provide
> in oal.document.
> Similarly, with LUCENE-3309, we've done the same thing on the
> doc/field retrieval side (from IndexReader), with the
> StoredFieldsVisitor.
> But, maybe we should break out StorableField from IndexableField,
> such that when you index a doc you provide two Iterables -- one for the
> IndexableFields and one for the StorableFields.  Either can be null.
> One downside is possible perf hit for fields that are both indexed &
> stored (ie, we visit them twice, lookup their name in a hash twice,
> etc.).  But the upside is a cleaner separation of concerns in API

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: [JENKINS] Lucene-trunk - Build # 1982 - Still Failing

[Uwe Schindler]

Hm, the Clover magic comment did not seem to help... I have no idea whats wrong 
with this test!

-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de


> -Original Message-
> From: Apache Jenkins Server [mailto:jenk...@builds.apache.org]
> Sent: Thursday, July 05, 2012 11:44 PM
> To: dev@lucene.apache.org
> Subject: [JENKINS] Lucene-trunk - Build # 1982 - Still Failing
> 
> Build: https://builds.apache.org/job/Lucene-trunk/1982/
> 
> All tests passed
> 
> Build Log:
> [...truncated 38248 lines...]
> 
> [...truncated 38248 lines...]
> 
> [...truncated 38248 lines...]
> 
> [...truncated 38248 lines...]
> 
> [...truncated 38248 lines...]
> 
> [...truncated 38248 lines...]
> 



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-trunk - Build # 1982 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-trunk/1982/

All tests passed

Build Log:
[...truncated 38248 lines...]

[...truncated 38248 lines...]

[...truncated 38248 lines...]

[...truncated 38248 lines...]

[...truncated 38248 lines...]

[...truncated 38248 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3312) Break out StorableField from IndexableField

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407509#comment-13407509
 ] 

Uwe Schindler commented on LUCENE-3312:
---

Hi Nikola,

I merged your patch to current tunk (removed lots of throws 
CorruptIndexException) and undid some formatti ng Changes in IndexReader.

Can you configure your IDE to *not* change unrelated code? This makes merging 
extremely hard.

I committed this in r1357938 to a new branch 
https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3312 (from trunk 
r1357904, Lucene 5.0). Please use this branch for further work! I will merge it 
regularily when bigger changes are in main dev, so please keep it updated. 
Please provide new patches against this branch.

> Break out StorableField from IndexableField
> ---
>
> Key: LUCENE-3312
> URL: https://issues.apache.org/jira/browse/LUCENE-3312
> Project: Lucene - Java
>  Issue Type: Improvement
>  Components: core/index
>Reporter: Michael McCandless
>Assignee: Nikola Tankovic
>  Labels: gsoc2012, lucene-gsoc-12
> Fix For: Field Type branch
>
> Attachments: lucene-3312-patch-01.patch, lucene-3312-patch-02.patch, 
> lucene-3312-patch-03.patch, lucene-3312-patch-04.patch, 
> lucene-3312-patch-05.patch, lucene-3312-patch-06.patch
>
>
> In the field type branch we have strongly decoupled
> Document/Field/FieldType impl from the indexer, by having only a
> narrow API (IndexableField) passed to IndexWriter.  This frees apps up
> use their own "documents" instead of the "user-space" impls we provide
> in oal.document.
> Similarly, with LUCENE-3309, we've done the same thing on the
> doc/field retrieval side (from IndexReader), with the
> StoredFieldsVisitor.
> But, maybe we should break out StorableField from IndexableField,
> such that when you index a doc you provide two Iterables -- one for the
> IndexableFields and one for the StorableFields.  Either can be null.
> One downside is possible perf hit for fields that are both indexed &
> stored (ie, we visit them twice, lookup their name in a hash twice,
> etc.).  But the upside is a cleaner separation of concerns in API

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-3355) Add shard name to SolrCore statistics

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller resolved SOLR-3355.
---

Resolution: Fixed

Thanks Michael!

> Add shard name to SolrCore statistics
> -
>
> Key: SOLR-3355
> URL: https://issues.apache.org/jira/browse/SOLR-3355
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Michael Garski
>Assignee: Mark Miller
>Priority: Trivial
> Fix For: 4.0, 5.0
>
> Attachments: SOLR-3355.patch, SOLR-3355.patch
>
>
> The JMX stats of the core do not expose the shard name that it is hosting, 
> which could be of use.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Linux-Java8-64 - Build # 8 - Still Failing!

[Policeman Jenkins Server]

Build: 
http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java8-64/8/

2 tests failed.
REGRESSION:  
org.apache.solr.handler.component.SpellCheckComponentTest.testPerDictionary

Error Message:
mismatch: '0'!='2' @ spellcheck/suggestions/bar/startOffset

Stack Trace:
java.lang.RuntimeException: mismatch: '0'!='2' @ 
spellcheck/suggestions/bar/startOffset
at 
__randomizedtesting.SeedInfo.seed([1532C642CF098CBF:D23C32EAC76599F7]:0)
at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:547)
at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:495)
at 
org.apache.solr.handler.component.SpellCheckComponentTest.testPerDictionary(SpellCheckComponentTest.java:102)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:474)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:818)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:877)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:825)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:671)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:697)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:736)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:747)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
at 
org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
at 
org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551)


FAILED:  
org.apache.solr.handler.component.DistributedQueryElevationComponentTest.testDistribSearch

Error Message:
Index: 18, Size: 18

Stack Trace:
java.lang.IndexOutO

Multi-thread UpdateProcessor

[Mikhail Khludnev]

Hello,

Most times when single thread streaming
http://wiki.apache.org/solr/Solrj#Streaming_documents_for_an_update is used
I saw lack of cpu utilization at Solr server. Resonable motivation is
utilize more threads to index faster, but it requires more complicated
 client side.
I propose to employ special update processor which can fork the stream
processing onto many threads. If you like it pls vote for
https://issues.apache.org/jira/browse/SOLR-3585 .

Regards

-- 
Sincerely yours
Mikhail Khludnev
Tech Lead
Grid Dynamics


 

[jira] [Commented] (LUCENE-4100) Maxscore - Efficient Scoring

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407472#comment-13407472
 ] 

Robert Muir commented on LUCENE-4100:
-

I spun off a sub-issue (LUCENE-4198) to see how we can first fix this Codec API 
so that
you don't need an IndexRewriter and this patch could work "live".

> Maxscore - Efficient Scoring
> 
>
> Key: LUCENE-4100
> URL: https://issues.apache.org/jira/browse/LUCENE-4100
> Project: Lucene - Java
>  Issue Type: Improvement
>  Components: core/codecs, core/query/scoring, core/search
>Affects Versions: 4.0
>Reporter: Stefan Pohl
>  Labels: api-change, patch, performance
> Fix For: 4.0
>
> Attachments: contrib_maxscore.tgz, maxscore.patch
>
>
> At Berlin Buzzwords 2012, I will be presenting 'maxscore', an efficient 
> algorithm first published in the IR domain in 1995 by H. Turtle & J. Flood, 
> that I find deserves more attention among Lucene users (and developers).
> I implemented a proof of concept and did some performance measurements with 
> example queries and lucenebench, the package of Mike McCandless, resulting in 
> very significant speedups.
> This ticket is to get started the discussion on including the implementation 
> into Lucene's codebase. Because the technique requires awareness about it 
> from the Lucene user/developer, it seems best to become a contrib/module 
> package so that it consciously can be chosen to be used.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4198) Allow codecs to index term impacts

[Robert Muir (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Muir updated LUCENE-4198:


Attachment: LUCENE-4198_flush.patch

here's a patch fixing how we compute stats in FreqProxTermsWriter: but the 
codec api is unchanged.

Next ill look at merge, which is trickier, and then see about changing the 
codec api.

> Allow codecs to index term impacts
> --
>
> Key: LUCENE-4198
> URL: https://issues.apache.org/jira/browse/LUCENE-4198
> Project: Lucene - Java
>  Issue Type: Sub-task
>  Components: core/index
>Reporter: Robert Muir
> Attachments: LUCENE-4198_flush.patch
>
>
> Subtask of LUCENE-4100.
> Thats an example of something similar to impact indexing (though, his 
> implementation currently stores a max for the entire term, the problem is the 
> same).
> We can imagine other similar algorithms too: I think the codec API should be 
> able to support these.
> Currently it really doesnt: Stefan worked around the problem by providing a 
> tool to 'rewrite' your index, he passes the IndexReader and Similarity to it. 
> But it would be better if we fixed the codec API.
> One problem is that the Postings writer needs to have access to the 
> Similarity. Another problem is that it needs access to the term and 
> collection statistics up front, rather than after the fact.
> This might have some cost (hopefully minimal), so I'm thinking to experiment 
> in a branch with these changes and see if we can make it work well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-4198) Allow codecs to index term impacts

[Robert Muir (JIRA)]

Robert Muir created LUCENE-4198:
---

 Summary: Allow codecs to index term impacts
 Key: LUCENE-4198
 URL: https://issues.apache.org/jira/browse/LUCENE-4198
 Project: Lucene - Java
  Issue Type: Sub-task
  Components: core/index
Reporter: Robert Muir


Subtask of LUCENE-4100.

Thats an example of something similar to impact indexing (though, his 
implementation currently stores a max for the entire term, the problem is the 
same).

We can imagine other similar algorithms too: I think the codec API should be 
able to support these.

Currently it really doesnt: Stefan worked around the problem by providing a 
tool to 'rewrite' your index, he passes the IndexReader and Similarity to it. 
But it would be better if we fixed the codec API.

One problem is that the Postings writer needs to have access to the Similarity. 
Another problem is that it needs access to the term and collection statistics 
up front, rather than after the fact.

This might have some cost (hopefully minimal), so I'm thinking to experiment in 
a branch with these changes and see if we can make it work well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting

[Trey Grainger (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407459#comment-13407459
 ] 

Trey Grainger commented on SOLR-2894:
-

Hi Erik,

Sorry, I missed your original message asking me if I could test out the latest 
patch - I'd be happy to help.  I just tried both your patch and the April 25th 
patch against the Solr 4.0 Alpha revision and neither applied immediately.  
I'll see if I can find some time on Sunday to try to get a revision sorted out 
which will work with the current version.

I think there are some changes in the April 24th patch which may need to be 
re-applied if your changes were based upon the earlier patch.  I'll know more 
once I've had a chance to dig in later this weekend.

Thanks,

-Trey

> Implement distributed pivot faceting
> 
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
>  Issue Type: Improvement
>Reporter: Erik Hatcher
>Assignee: Erik Hatcher
> Fix For: 4.0
>
> Attachments: SOLR-2894.patch, SOLR-2894.patch, 
> distributed_pivot.patch, distributed_pivot.patch
>
>
> Following up on SOLR-792, pivot faceting currently only supports 
> undistributed mode.  Distributed pivot faceting needs to be implemented.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3594) SolrCore() doesn't wait SolrCore.getSearcher() to register _searcher

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407416#comment-13407416
 ] 

Hoss Man commented on SOLR-3594:


bq. The real question here is unrelated to tests: should the SolrCore 
constructor should wait for a searcher to be registered before returning?

i don't think so. Just because a searcher isn't available yet, doesn't mean the 
SolrCore is unusable - we shouldn't block other uses of the SolrCore just 
because a searcher isn't available yet. the first thread that attempts to use 
getSearcher() is what should block on the listeners (depending on the setting 
of useColdSearcher)

The test failure suggests to me that something is wonky with how were are 
tracking the searcher opens and doing cleanup -- either in SolrCore.close() or 
in the test framework itself.

> SolrCore() doesn't wait SolrCore.getSearcher() to register _searcher
> 
>
> Key: SOLR-3594
> URL: https://issues.apache.org/jira/browse/SOLR-3594
> Project: Solr
>  Issue Type: Bug
>  Components: clients - java
>Affects Versions: 3.4
>Reporter: Egor Pahomov
>Priority: Minor
>  Labels: test
> Attachments: 3594.patch, testSearchersManagement.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> SolrCore() executes SolrCore.getSearcher(...) and returns without checking if 
> getSearcher(...) already registered _searcher. As result: if we have 
> SolrEventListener with slow newSearcher(), we can end test before _searcher 
> registered and get then searcher closes and searcher opens doesn't match. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: jira tracking of issues fixed in 4.0-ALPHA

[Uwe Schindler]

+1, as it was my idea :)
--
Uwe Schindler
H.-H.-Meier-Allee 63, 28213 Bremen
http://www.thetaphi.de



Chris Hostetter  schrieb:


I've seen no comments on this: if anyone objects please speak up or i'll 
move forward with this soon.

: : I think for this case, the much easier fix would be to rename the 4.0 
: : version to 4.0-alpha and create a new 4.0 one. All not yet fixed would 
: : get this new version as fix version.
: 
: Doh! ... why didn't i think of that?
: 
: Anybody object to this sequence?
: 
: In Jira project admin for both SOLR and LUCENE...
: 1) delete version "4.0-ALPHA" (it has no issues yet in either project)
: 2) rename version "4.0" to "4.0-ALPHA"
: 3) add a new version "4.0"
: 
: if/when we get to 4.0-BETA we can do the same thing


-Hoss

_

To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Solr 3.6.0 javadocs are missing from the site

[Chris Hostetter]

: thats the whole problem with /api: its not defined at all.

it has bee nvery clearly defined since it was created: "the latest 
javadocs" ... just because we no longer explicitly link to it, doesn't 
mean we should stop trying to live up to the point of the link -- 
especially not when it's so fucking easy to do.

: having shit like this just turns into 'lets blame the release manager
: when things change and its not the way i want'.

where do you get that anyone is going to blame release managers for 
something?  having this redirect isn't going to break anything, nor does 
it have anything to do with anything an RM should give a fuck about.

the only reason we had a hicup with it on tuesday was because that was the 
day we made the change from only hosting single copy of the solr javadocs, 
re-using a single path for each new version, to having multiple versions 
with distinct pathes -- and when we made that change we did *NOT* have any 
redirect like this in place at all.

that change could have been made at any time, regardless of wether it 
involved a new release, regardless of wether it was done by an RM, and the 
problem would have been the same: the missing redirect ment old links 
broke.

that was a one time change, that will never affect any other release in 
the future ever again: we just keep adding new directories for the new 
docs.

having this redirect doesn't affect that in any way shape or form

: same goes for download redirect links (I will open an issue tomorrow:
: either we remove these download redirect llinks completely, or we fix
: them to take versions, because having to add ?'s with bogus stuff on

I already opened an issue for that when we noticed this during 3.6 .. no 
one who cares about the google analytics and understands javascript has 
bothered to pick it up...

https://issues.apache.org/jira/browse/LUCENE-3978


-Hoss

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: jira tracking of issues fixed in 4.0-ALPHA

[Chris Hostetter]

I've seen no comments on this: if anyone objects please speak up or i'll 
move forward with this soon.

: : I think for this case, the much easier fix would be to rename the 4.0 
: : version to 4.0-alpha and create a new 4.0 one. All not yet fixed would 
: : get this new version as fix version.
: 
: Doh! ... why didn't i think of that?
: 
: Anybody object to this sequence?
: 
: In Jira project admin for both SOLR and LUCENE...
:  1) delete version "4.0-ALPHA" (it has no issues yet in either project)
:  2) rename version "4.0" to "4.0-ALPHA"
:  3) add a new version "4.0"
: 
: if/when we get to 4.0-BETA we can do the same thing


-Hoss

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4191) Lucene doc pages redirect to "api-4_0_0-ALPHA" which results in 404

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407386#comment-13407386
 ] 

Hoss Man commented on LUCENE-4191:
--

BaseTokenFilterFactory no longer exists in the latest version of Solr (most of 
the Factory concepts were refactored up into the Lucene-Core analysis-common 
module) and Google has not yet updated it's crawl of solr javadocs.

Solr 3.6 javadocs are still available, or you can follow links from he Solr 
4.0-ALPHA javadocs over to the Lucene-Core javadocs for classes like 
TokenFilterFactory and AbstractAnalysisFactory ...

http://lucene.apache.org/solr/documentation.html


> Lucene doc pages redirect to "api-4_0_0-ALPHA" which results in 404
> ---
>
> Key: LUCENE-4191
> URL: https://issues.apache.org/jira/browse/LUCENE-4191
> Project: Lucene - Java
>  Issue Type: Bug
>Affects Versions: 3.6
>Reporter: Chaim Peck
>  Labels: documentation
>
> Try to go to this URL:
> http://lucene.apache.org/solr/api/org/apache/solr/analysis/BaseTokenFilterFactory.html
> The result is that you will be redirected here, which is a 404:
> http://lucene.apache.org/solr/api-4_0_0-ALPHA/org/apache/solr/analysis/BaseTokenFilterFactory.html
> You can still get to the page from google cache:
> http://webcache.googleusercontent.com/search?q=cache:mCJCac4iZ0QJ:lucene.apache.org/solr/api/org/apache/solr/analysis/BaseTokenFilterFactory.html+&cd=1&hl=en&ct=clnk&gl=us

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-3593) add "/solr/api/index.html"

[Hoss Man (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man resolved SOLR-3593.


Resolution: Fixed

FWIW: i went ahead and used the "documentation.html" name after all because i 
realized it kept hte page editing simpler.  it was easy to deal with the legacy 
"/solr/api", "/solr/api/" and "/solr/api/index.html" type top level links using 
redirects...



> add "/solr/api/index.html"
> --
>
> Key: SOLR-3593
> URL: https://issues.apache.org/jira/browse/SOLR-3593
> Project: Solr
>  Issue Type: Bug
>Reporter: Hoss Man
>Assignee: Hoss Man
>
> solr historically only had one version of the javadocs on the site at a time.
> particularly now that we have 3.6.X and 4.X concurrently, this needs to 
> change.  both sets of javadoc are already on the site, and 
> "/solr/tutorial.html" already links to both versions appropriately but there 
> are still some improvements that should be made...
> * add a "/solr/api/index.html" file that mirrors the type of inof listed on 
> /core/documentation.html
> ** we could use the same "documentation.html" name, but since historically 
> lots of people have bookmarked/linked to "/solr/api" reusing that path as the 
> "landing" page for finding docs about multiple versions seems better
> ** making this visible will probably mean needing to dial in the existing 
> /solr/api redirect more
> * update the "Javadocs" link i nthe right nav to link to this page

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4190) IndexWriter deletes non-Lucene files

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407365#comment-13407365
 ] 

Robert Muir commented on LUCENE-4190:
-

{quote}
I think that the way to "bound" the namespace of files is to put everything in 
a subdirectory of the index directory chosen by the user and control the name 
of that subdirectory, making it clear that this is semi-private to Lucene and 
that all files in that subdirectory are fair game.
{quote}

Well there are a couple challenges with that I think:
1. subdirectories currently are a foreign concept to Directory, we would have 
to make some serious changes there to support subdirectories.
2. Lucene 3.x and Lucene4-alpha indexes still need to be supported, and we dont 
want to leave behind baggage when we merge, so the transition would be tricky.
3. the user could also do this on their own right? e.g. we still have the same 
situation we have currently, where anything in that directory can get deleted 
by lucene, its just underneath another layer.


> IndexWriter deletes non-Lucene files
> 
>
> Key: LUCENE-4190
> URL: https://issues.apache.org/jira/browse/LUCENE-4190
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Robert Muir
> Fix For: 4.0, 5.0
>
> Attachments: LUCENE-4190.patch, LUCENE-4190.patch, LUCENE-4190.patch
>
>
> Carl Austin raised a good issue in a comment on my Lucene 4.0.0 alpha blog 
> post: 
> http://blog.mikemccandless.com/2012/07/lucene-400-alpha-at-long-last.html
> IndexWriter will now (as of 4.0) delete all foreign files from the index 
> directory.  We made this change because Codecs are free to write to any files 
> now, so the space of filenames is hard to "bound".
> But if the user accidentally uses the wrong directory (eg c:/) then we will 
> in fact delete important stuff.
> I think we can at least use some simple criteria (must start with _, maybe 
> must fit certain pattern eg _(_X).Y), so we are much less likely to 
> delete a non-Lucene file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4197) Small improvements to Lucene Spatial Module for v4

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley updated LUCENE-4197:
-

Attachment: 
LUCENE-4197_SpatialArgs_doesn_t_need_overloaded_toString()_with_a_ctx_param_.patch

SpatialArgs.toString() shouldn't be overloaded with a ctx -- not needed for its 
purpose.  Nobody was calling it any way.  What instigated this finding was that 
this class depended on SimpleSpatialContext, gone in 0.3-SNAPSHOT of Spatial4j.

> Small improvements to Lucene Spatial Module for v4
> --
>
> Key: LUCENE-4197
> URL: https://issues.apache.org/jira/browse/LUCENE-4197
> Project: Lucene - Java
>  Issue Type: Improvement
>  Components: modules/spatial
>Reporter: David Smiley
> Fix For: 4.0
>
> Attachments: 
> LUCENE-4197_SpatialArgs_doesn_t_need_overloaded_toString()_with_a_ctx_param_.patch
>
>
> This issue is to capture small changes to the Lucene spatial module that 
> don't deserve their own issue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-4197) Small improvements to Lucene Spatial Module for v4

[David Smiley (JIRA)]

David Smiley created LUCENE-4197:


 Summary: Small improvements to Lucene Spatial Module for v4
 Key: LUCENE-4197
 URL: https://issues.apache.org/jira/browse/LUCENE-4197
 Project: Lucene - Java
  Issue Type: Improvement
  Components: modules/spatial
Reporter: David Smiley
 Fix For: 4.0


This issue is to capture small changes to the Lucene spatial module that don't 
deserve their own issue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-tests-only-4.x - Build # 207 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-tests-only-4.x/207/

3 tests failed.
REGRESSION:  
org.apache.solr.handler.dataimport.TestSqlEntityProcessorDelta3.testCompositePk_DeltaImport_delete

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
at 
__randomizedtesting.SeedInfo.seed([2AB24089FB28443:2396D216FACFAC2F]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:461)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:428)
at 
org.apache.solr.handler.dataimport.TestSqlEntityProcessorDelta3.testCompositePk_DeltaImport_delete(TestSqlEntityProcessorDelta3.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:818)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:877)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:825)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:671)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:697)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:736)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:747)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
at 
org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
at 
org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//*[@numFound='0']
xml response was: 

000*:* OR testCompositePk_DeltaImport_deletestandard202.2d122012-07-05T16:51:24.176Z


request 
w

[jira] [Updated] (SOLR-3355) Add shard name to SolrCore statistics

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-3355:
--

Attachment: SOLR-3355.patch

> Add shard name to SolrCore statistics
> -
>
> Key: SOLR-3355
> URL: https://issues.apache.org/jira/browse/SOLR-3355
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Michael Garski
>Assignee: Mark Miller
>Priority: Trivial
> Fix For: 4.0, 5.0
>
> Attachments: SOLR-3355.patch, SOLR-3355.patch
>
>
> The JMX stats of the core do not expose the shard name that it is hosting, 
> which could be of use.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3355) Add shard name to SolrCore statistics

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-3355:
--

Fix Version/s: 5.0

I've added collection as well and wrote a couple tests for this - I'll commit 
shortly.

> Add shard name to SolrCore statistics
> -
>
> Key: SOLR-3355
> URL: https://issues.apache.org/jira/browse/SOLR-3355
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Michael Garski
>Assignee: Mark Miller
>Priority: Trivial
> Fix For: 4.0, 5.0
>
> Attachments: SOLR-3355.patch
>
>
> The JMX stats of the core do not expose the shard name that it is hosting, 
> which could be of use.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-4.x-Linux-Java8-64 - Build # 5 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux-Java8-64/5/

1 tests failed.
REGRESSION:  
org.apache.solr.handler.component.SpellCheckComponentTest.testPerDictionary

Error Message:
mismatch: '0'!='2' @ spellcheck/suggestions/bar/startOffset

Stack Trace:
java.lang.RuntimeException: mismatch: '0'!='2' @ 
spellcheck/suggestions/bar/startOffset
at 
__randomizedtesting.SeedInfo.seed([9AA8B04990EA81A6:5DA644E1988694EE]:0)
at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:547)
at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:495)
at 
org.apache.solr.handler.component.SpellCheckComponentTest.testPerDictionary(SpellCheckComponentTest.java:102)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:474)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:818)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:877)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:825)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:671)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:697)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:736)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:747)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
at 
org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
at 
org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551)




Build Log:
[...truncated 8725 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java8-64/checkout/build.xml:29:
 The following error occurred while e

Re: Question about solr config files encoding.

[Dawid Weiss]

Sure, I don't have a problem with XML. I'll assume UTF-8 for json and
go through the issues later today.

Dawid

On Thu, Jul 5, 2012 at 5:47 PM, Uwe Schindler  wrote:
> I just add:
>
> Solr's XML files are parsed according to XML spec, so you can choose any
> charset, you only have to define it according to XML spec! Also XML POST to
> updatehandler can be any encoding (it does not need to be declared in header
> anymore, the  header is fine). There is already a test! I Fixed all
> this in endless sessions, but I was happy to do it, as my favourite data
> format is: XML :-) [I refuse to fix this for DIH, but that's another story,
> SOLR-2347].
>
> Uwe
>
> -
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: u...@thetaphi.de
>
>
>> -Original Message-
>> From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik
>> Seeley
>> Sent: Thursday, July 05, 2012 5:43 PM
>> To: dev@lucene.apache.org
>> Subject: Re: Question about solr config files encoding.
>>
>> On Thu, Jul 5, 2012 at 10:59 AM, Dawid Weiss 
>> wrote:
>> > According to JSON RFC:
>> >
>> > http://tools.ietf.org/html/rfc4627#section-3
>> >
>> > JSON text SHALL be encoded in Unicode.
>>
>> One of my little pet peeves with the RFC - I think this was a bad
> requirement.
>> JSON should have been text, and then their should have been an optional
> way
>> to detect encoding if other mechanisms don't cover it (like HTTP headers,
> etc).
>> This effectively means that something like ["hi"] is not valid JSON for
> many of
>> you reading this email (if your email client is internally representing it
> as
>> something other than unicode encoded for example).
>>
>>
>> > We could just enforce/require UTF-8?
>>
>> Yes, Solr has normally always required/assumed UTF-8 for config files.
>>  It's simply an oversight in any places that don't.
>>
>> -Yonik
>> http://lucidimagination.com
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional
>> commands, e-mail: dev-h...@lucene.apache.org
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: Question about solr config files encoding.

[Uwe Schindler]

> updatehandler can be any encoding (it does not need to be declared in
header

...HTTP header..., sorry

> > -Original Message-
> > From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik
> > Seeley
> > Sent: Thursday, July 05, 2012 5:43 PM
> > To: dev@lucene.apache.org
> > Subject: Re: Question about solr config files encoding.
> >
> > On Thu, Jul 5, 2012 at 10:59 AM, Dawid Weiss 
> > wrote:
> > > According to JSON RFC:
> > >
> > > http://tools.ietf.org/html/rfc4627#section-3
> > >
> > > JSON text SHALL be encoded in Unicode.
> >
> > One of my little pet peeves with the RFC - I think this was a bad
> requirement.
> > JSON should have been text, and then their should have been an
> > optional
> way
> > to detect encoding if other mechanisms don't cover it (like HTTP
> > headers,
> etc).
> > This effectively means that something like ["hi"] is not valid JSON
> > for
> many of
> > you reading this email (if your email client is internally
> > representing it
> as
> > something other than unicode encoded for example).
> >
> >
> > > We could just enforce/require UTF-8?
> >
> > Yes, Solr has normally always required/assumed UTF-8 for config files.
> >  It's simply an oversight in any places that don't.
> >
> > -Yonik
> > http://lucidimagination.com
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For
> > additional commands, e-mail: dev-h...@lucene.apache.org
> 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional
> commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4188) Storing Shapes shouldn't be Strategy dependent

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407221#comment-13407221
 ] 

David Smiley commented on LUCENE-4188:
--

RE createStoredField():
bq. I don't really like this. It is barely an improvement on the current code. 
The whole point of this issue is that the storing of Shapes shouldn't be 
related to Strategys. I think we should be explicit and require the consumer 
code (Solr or something else) decides how it wants to store Shapes. If you want 
a convenience method then it should be static, illustrating it is a utility 
that the Strategys cannot override. Ideally I would like it somewhere else 
entirely.

The client doesn't have to use this method, but in all tests + the Solr 
adapters I don't think there's a reason not to.  I found it to be useful, and 
to provide a place to document how it is recommended to store the shape (notice 
I even included the one-liner source in the javadocs).  An advantage of it 
being an instance method on the Strategy is that it has convenient access to 
both the field name & SpatialContext.  I could make this method final, and I 
could add more documentation that makes it clear that the user is free to store 
the shape in any way they wish since the spatial module doesn't care.

> Storing Shapes shouldn't be Strategy dependent
> --
>
> Key: LUCENE-4188
> URL: https://issues.apache.org/jira/browse/LUCENE-4188
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: modules/spatial
>Reporter: Chris Male
>Assignee: David Smiley
> Attachments: LUCENE-4188_remove_field_storage_from_createField.patch
>
>
> The logic for storing Shape representations seems to be different for each 
> Strategy.  The PrefixTreeStrategy impls store the Shape in WKT, which is nice 
> if you're using WKT but not much help if you're not.  BBoxStrategy doesn't 
> actually store the Shape itself, but a representation of the bounding box.  
> TwoDoubles seems to follow the PrefixTreeStrategy approach, which is 
> surprising since it only indexes Points and they could be stored without 
> using WKT.
> I think we need to consider what storing a Shape means.  If we want to store 
> the Shape itself, then that logic should be standardised and done outside of 
> the Strategys since it is not really related to them.  If we want to store 
> the terms being used by the Strategys to make Shapes queryable, then we need 
> to change the logic in the Strategys to actually do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: Question about solr config files encoding.

[Uwe Schindler]

I just add:

Solr's XML files are parsed according to XML spec, so you can choose any
charset, you only have to define it according to XML spec! Also XML POST to
updatehandler can be any encoding (it does not need to be declared in header
anymore, the  header is fine). There is already a test! I Fixed all
this in endless sessions, but I was happy to do it, as my favourite data
format is: XML :-) [I refuse to fix this for DIH, but that's another story,
SOLR-2347].

Uwe

-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de


> -Original Message-
> From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik
> Seeley
> Sent: Thursday, July 05, 2012 5:43 PM
> To: dev@lucene.apache.org
> Subject: Re: Question about solr config files encoding.
> 
> On Thu, Jul 5, 2012 at 10:59 AM, Dawid Weiss 
> wrote:
> > According to JSON RFC:
> >
> > http://tools.ietf.org/html/rfc4627#section-3
> >
> > JSON text SHALL be encoded in Unicode.
> 
> One of my little pet peeves with the RFC - I think this was a bad
requirement.
> JSON should have been text, and then their should have been an optional
way
> to detect encoding if other mechanisms don't cover it (like HTTP headers,
etc).
> This effectively means that something like ["hi"] is not valid JSON for
many of
> you reading this email (if your email client is internally representing it
as
> something other than unicode encoded for example).
> 
> 
> > We could just enforce/require UTF-8?
> 
> Yes, Solr has normally always required/assumed UTF-8 for config files.
>  It's simply an oversight in any places that don't.
> 
> -Yonik
> http://lucidimagination.com
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional
> commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Question about solr config files encoding.

[Yonik Seeley]

On Thu, Jul 5, 2012 at 10:59 AM, Dawid Weiss  wrote:
> According to JSON RFC:
>
> http://tools.ietf.org/html/rfc4627#section-3
>
> JSON text SHALL be encoded in Unicode.

One of my little pet peeves with the RFC - I think this was a bad
requirement.  JSON should have been text, and then their should have
been an optional way to detect encoding if other mechanisms don't
cover it (like HTTP headers, etc).  This effectively means that
something like
["hi"] is not valid JSON for many of you reading this email (if your
email client is internally representing it as something other than
unicode encoded for example).


> We could just enforce/require UTF-8?

Yes, Solr has normally always required/assumed UTF-8 for config files.
 It's simply an oversight in any places that don't.

-Yonik
http://lucidimagination.com

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: Question about solr config files encoding.

[Uwe Schindler]

3.  Encoding

   JSON text SHALL be encoded in Unicode.  The default encoding is
   UTF-8.

   Since the first two characters of a JSON text will always be ASCII
   characters [RFC0020], it is possible to determine whether an octet
   stream is UTF-8, UTF-16 (BE or LE), or UTF-32 (BE or LE) by looking
   at the pattern of nulls in the first four octets.

   00 00 00 xx  UTF-32BE
   00 xx 00 xx  UTF-16BE
   xx 00 00 00  UTF-32LE
   xx 00 xx 00  UTF-16LE
   xx xx xx xx  UTF-8

:-)

I think we can safely assume it is UTF-8, otherwise we must do the same shit 
like XML parsers with mark() on BufferedInputStream Most libraries out 
there can only read UTF-8 and SOLR itself produces only UTF8 JSON, right? Those 
tests only check response from solr.

Uwe

-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de


> -Original Message-
> From: dawid.we...@gmail.com [mailto:dawid.we...@gmail.com] On Behalf Of
> Dawid Weiss
> Sent: Thursday, July 05, 2012 5:35 PM
> To: dev@lucene.apache.org
> Subject: Re: Question about solr config files encoding.
> 
> > But JSON is defined to be UTF-8, so we must supply the encoding
> (IOUtils.UTF8_CHARSET).
> 
> That RFC says it can be any unicode... this said I agree with you that we can
> probably assume it's UTF-8 and not worry about anything else.
> 
> Dawid
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional
> commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Question about solr config files encoding.

[Dawid Weiss]

> But JSON is defined to be UTF-8, so we must supply the encoding 
> (IOUtils.UTF8_CHARSET).

That RFC says it can be any unicode... this said I agree with you that
we can probably assume it's UTF-8 and not worry about anything else.

Dawid

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4196) Turn asserts in I/O related code into hard checks

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407207#comment-13407207
 ] 

Robert Muir commented on LUCENE-4196:
-

That one is a good example of something we should watch out for, i think its ok 
because it uses IndexInput.length,
but we should make sure we don't directly turn asserts that use things like 
Directory.fileExists or Directory.fileLength into
real checks, it could cause problems for NFS (LUCENE-3727)



> Turn asserts in I/O related code into hard checks
> -
>
> Key: LUCENE-4196
> URL: https://issues.apache.org/jira/browse/LUCENE-4196
> Project: Lucene - Java
>  Issue Type: Task
>  Components: core/index
>Affects Versions: 4.0-ALPHA
>Reporter: Uwe Schindler
> Fix For: 4.0
>
>
> In lots of codecs we only assert, that e.g. some things inside files are 
> correctly in bounds, which leads to security problems (ok, not as bad as 
> C-Style buffer overflows), but e.g. allocating a large array after reading a 
> VInt from a file header and then OOM, is a security issue. So we have to 
> check all those contracts for files as hard checks, especially as a simply 
> check in most cases dont cost anything (and it costs not more than the assert 
> itsself, as the assert also takes CPU power, because it needs a check one 
> time on a static final class field).
> Of course we cannot check values we read when reading postings, but the 
> simple checks that any postings file has correct header and something like a 
> positive number of elements, or number of elements < file size,..., a 
> bit-fireld only contains valid bits in StoredFieldsReader, or non-duplicate 
> filenames (CFS) are very important. We had those checks in 3.x, but in 4.0, 
> Mike changed all of those to asserts during the flex development (in my 
> opinion with no real reason).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: Question about solr config files encoding.

[Uwe Schindler]

Config fiules are XML and I changed them to be handled by the XML parser 
(InputStreams), so XML parser reads encoding from Header.

But JSON is defined to be UTF-8, so we must supply the encoding 
(IOUtils.UTF8_CHARSET).

-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de


> -Original Message-
> From: Dawid Weiss [mailto:dawid.we...@gmail.com]
> Sent: Thursday, July 05, 2012 5:00 PM
> To: dev@lucene.apache.org
> Subject: Question about solr config files encoding.
> 
> Guys should the encoding of config files really be platform-dependent?
> Currently Solr tests fail massively on setup because of things like
> this:
> 
> public OpenExchangeRates(InputStream ratesStream) throws IOException {
>   parser = new JSONParser(new InputStreamReader(ratesStream));
> 
> this reader, when confronted with UTF-16 as file.encoding results in funky
> exceptions like:
> 
>> Caused by: org.apache.noggit.JSONParser$ParseException: JSON Parse
> Error: char=笊,position=0 BEFORE='笊'
> AFTER='†≤楳捬慩浥爢㨠≔桩猠摡瑡⁩猠捯汬散瑥搠晲潭⁶慲楯畳⁰牯癩摥牳⁡
> 湤⁰牯癩摥搠晲'
>>  at org.apache.noggit.JSONParser.err(JSONParser.java:221)
>>  at org.apache.noggit.JSONParser.next(JSONParser.java:620)
>>  at org.apache.noggit.JSONParser.nextEvent(JSONParser.java:661)
>>  at
> org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates.
> (OpenExchangeRatesOrgProvider.java:189)
>>  at
> org.apache.solr.schema.OpenExchangeRatesOrgProvider.reload(OpenExchang
> eRatesOrgProvider.java:129)
> 
> Can we fix the encoding of these input files to UTF-8 or something?
> According to JSON RFC:
> 
> http://tools.ietf.org/html/rfc4627#section-3
> 
> JSON text SHALL be encoded in Unicode.  The default encoding is
>UTF-8.
> 
>Since the first two characters of a JSON text will always be ASCII
>characters [RFC0020], it is possible to determine whether an octet
>stream is UTF-8, UTF-16 (BE or LE), or UTF-32 (BE or LE) by looking
>at the pattern of nulls in the first four octets.
> 
>00 00 00 xx  UTF-32BE
>00 xx 00 xx  UTF-16BE
>xx 00 00 00  UTF-32LE
>xx 00 xx 00  UTF-16LE
>xx xx xx xx  UTF-8
> 
> We could just enforce/require UTF-8? Alternatively, auto-detect this from a
> binary stream as a custom Reader class.
> 
> Dawid
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional
> commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4196) Turn asserts in I/O related code into hard checks

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407194#comment-13407194
 ] 

Robert Muir commented on LUCENE-4196:
-

I agree, sometimes I added asserts that I feel should be real checks but I feel 
its safest to just do the assert.

Lucene3xNormsProducer:119 is an example. if it fails it means you have a 
corrumpt .nrm file with wrong norms mismatched
for different fields.

> Turn asserts in I/O related code into hard checks
> -
>
> Key: LUCENE-4196
> URL: https://issues.apache.org/jira/browse/LUCENE-4196
> Project: Lucene - Java
>  Issue Type: Task
>  Components: core/index
>Affects Versions: 4.0-ALPHA
>Reporter: Uwe Schindler
> Fix For: 4.0
>
>
> In lots of codecs we only assert, that e.g. some things inside files are 
> correctly in bounds, which leads to security problems (ok, not as bad as 
> C-Style buffer overflows), but e.g. allocating a large array after reading a 
> VInt from a file header and then OOM, is a security issue. So we have to 
> check all those contracts for files as hard checks, especially as a simply 
> check in most cases dont cost anything (and it costs not more than the assert 
> itsself, as the assert also takes CPU power, because it needs a check one 
> time on a static final class field).
> Of course we cannot check values we read when reading postings, but the 
> simple checks that any postings file has correct header and something like a 
> positive number of elements, or number of elements < file size,..., a 
> bit-fireld only contains valid bits in StoredFieldsReader, or non-duplicate 
> filenames (CFS) are very important. We had those checks in 3.x, but in 4.0, 
> Mike changed all of those to asserts during the flex development (in my 
> opinion with no real reason).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4190) IndexWriter deletes non-Lucene files

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407196#comment-13407196
 ] 

Uwe Schindler commented on LUCENE-4190:
---

New patch looks good, I was not aware that the previous one was iterating over 
all files each time. As the SegmentInfo internal list should not be available 
outside, we have no problem anybody else changing this uncontrolled.

See also issue LUCENE-4196.

> IndexWriter deletes non-Lucene files
> 
>
> Key: LUCENE-4190
> URL: https://issues.apache.org/jira/browse/LUCENE-4190
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Robert Muir
> Fix For: 4.0, 5.0
>
> Attachments: LUCENE-4190.patch, LUCENE-4190.patch, LUCENE-4190.patch
>
>
> Carl Austin raised a good issue in a comment on my Lucene 4.0.0 alpha blog 
> post: 
> http://blog.mikemccandless.com/2012/07/lucene-400-alpha-at-long-last.html
> IndexWriter will now (as of 4.0) delete all foreign files from the index 
> directory.  We made this change because Codecs are free to write to any files 
> now, so the space of filenames is hard to "bound".
> But if the user accidentally uses the wrong directory (eg c:/) then we will 
> in fact delete important stuff.
> I think we can at least use some simple criteria (must start with _, maybe 
> must fit certain pattern eg _(_X).Y), so we are much less likely to 
> delete a non-Lucene file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-4196) Turn asserts in I/O related code into hard checks

[Uwe Schindler (JIRA)]

Uwe Schindler created LUCENE-4196:
-

 Summary: Turn asserts in I/O related code into hard checks
 Key: LUCENE-4196
 URL: https://issues.apache.org/jira/browse/LUCENE-4196
 Project: Lucene - Java
  Issue Type: Task
  Components: core/index
Affects Versions: 4.0-ALPHA
Reporter: Uwe Schindler
 Fix For: 4.0


In lots of codecs we only assert, that e.g. some things inside files are 
correctly in bounds, which leads to security problems (ok, not as bad as 
C-Style buffer overflows), but e.g. allocating a large array after reading a 
VInt from a file header and then OOM, is a security issue. So we have to check 
all those contracts for files as hard checks, especially as a simply check in 
most cases dont cost anything (and it costs not more than the assert itsself, 
as the assert also takes CPU power, because it needs a check one time on a 
static final class field).
Of course we cannot check values we read when reading postings, but the simple 
checks that any postings file has correct header and something like a positive 
number of elements, or number of elements < file size,..., a bit-fireld only 
contains valid bits in StoredFieldsReader, or non-duplicate filenames (CFS) are 
very important. We had those checks in 3.x, but in 4.0, Mike changed all of 
those to asserts during the flex development (in my opinion with no real 
reason).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4190) IndexWriter deletes non-Lucene files

[Robert Muir (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Muir updated LUCENE-4190:


Attachment: LUCENE-4190.patch

updated patch with _ check turned into a hard check.

> IndexWriter deletes non-Lucene files
> 
>
> Key: LUCENE-4190
> URL: https://issues.apache.org/jira/browse/LUCENE-4190
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Robert Muir
> Fix For: 4.0, 5.0
>
> Attachments: LUCENE-4190.patch, LUCENE-4190.patch, LUCENE-4190.patch
>
>
> Carl Austin raised a good issue in a comment on my Lucene 4.0.0 alpha blog 
> post: 
> http://blog.mikemccandless.com/2012/07/lucene-400-alpha-at-long-last.html
> IndexWriter will now (as of 4.0) delete all foreign files from the index 
> directory.  We made this change because Codecs are free to write to any files 
> now, so the space of filenames is hard to "bound".
> But if the user accidentally uses the wrong directory (eg c:/) then we will 
> in fact delete important stuff.
> I think we can at least use some simple criteria (must start with _, maybe 
> must fit certain pattern eg _(_X).Y), so we are much less likely to 
> delete a non-Lucene file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Windows-Java6-64 - Build # 696 - Failure!

[Policeman Jenkins Server]

Build: 
http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Windows-Java6-64/696/

1 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
ERROR: SolrIndexSearcher opens=74 closes=72

Stack Trace:
java.lang.AssertionError: ERROR: SolrIndexSearcher opens=74 closes=72
at __randomizedtesting.SeedInfo.seed([8F8AC7455F54278A]:0)
at org.junit.Assert.fail(Assert.java:93)
at 
org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:191)
at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:82)
at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:754)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
at 
org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
at 
org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551)




Build Log:
[...truncated 7663 lines...]
[junit4:junit4]   2> 161993 T2156 oas.SolrTestCaseJ4.endTrackingSearchers 
SEVERE ERROR: SolrIndexSearcher opens=74 closes=72
[junit4:junit4]   2> NOTE: test params are: codec=Appending, 
sim=RandomSimilarityProvider(queryNorm=false,coord=false): {}, locale=lv, 
timezone=Africa/Blantyre
[junit4:junit4]   2> NOTE: Windows 7 6.1 amd64/Sun Microsystems Inc. 1.6.0_32 
(64-bit)/cpus=2,threads=1,free=53764744,total=238157824
[junit4:junit4]   2> NOTE: All tests run in this JVM: 
[TestGermanLightStemFilterFactory, TestPropInjectDefaults, 
SolrCmdDistributorTest, TestTrimFilterFactory, TestSolrQueryParser, 
LukeRequestHandlerTest, SolrIndexConfigTest, PrimitiveFieldTypeTest, 
DistanceFunctionTest, MultiTermTest, TestThaiWordFilterFactory, 
SpatialFilterTest, FileBasedSpellCheckerTest, LengthFilterTest, 
TestRemoteStreaming, TestSort, PrimUtilsTest, TestConfig, 
XmlUpdateRequestHandlerTest, XsltUpdateRequestHandlerTest, 
LeaderElectionIntegrationTest, SolrCoreCheckLockOnStartupTest, 
TestKeywordMarkerFilterFactory, TestCJKWidthFilterFactory, TestCodecSupport, 
TestEnglishMinimalStemFilterFactory, FieldAnalysisRequestHandlerTest, 
FieldMutatingUpdateProcessorTest, TestDelimitedPayloadTokenFilterFactory, 
TestSwedishLightStemFilterFactory, TestSearchPerf, TestBinaryField, 
BadIndexSchemaTest, TestTurkishLowerCaseFilterFactory, TestDistributedSearch, 
TestMappingCharFilterFactory, TestKeepFilterFactory, TestFaceting, 
FullSolrCloudTest, IndexBasedSpellCheckerTest, IndexSchemaTest, 
IndexReaderFactoryTest, TestHashPartitioner, TestIndexingPerformance, 
CoreAdminHandlerTest, SearchHandlerTest, 
TestHyphenationCompoundWordTokenFilterFactory, TestPropInject, 
TestFrenchMinimalStemFilterFactory, TestPortugueseMinimalStemFilterFactory, 
TestElisionFilterFactory, TestCapitalizationFilterFactory, 
TestItalianLightStemFilterFactory, SnowballPorterFilterFactoryTest, 
TestQuerySenderNoQuery, TestHindiFilters, TestStandardFactories, 
ZkSolrClientTest, TestCJKBigramFilterFactory, CloudStateUpdateTest, 
TestDFRSimilarityFactory, RAMDirectoryFactoryTest, 
OpenExchangeRatesOrgProviderTest, TestDefaultSimilarityFactory, 
TestKStemFilterFactory, TestGermanMinimal

[jira] [Commented] (LUCENE-4190) IndexWriter deletes non-Lucene files

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407185#comment-13407185
 ] 

Robert Muir commented on LUCENE-4190:
-

Uwe, no i mean that we check the entire list each time.

So if someone were to call addFile(), addFile(), addFile() that would be very 
bad runtime. Ill update the patch.

> IndexWriter deletes non-Lucene files
> 
>
> Key: LUCENE-4190
> URL: https://issues.apache.org/jira/browse/LUCENE-4190
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Robert Muir
> Fix For: 4.0, 5.0
>
> Attachments: LUCENE-4190.patch, LUCENE-4190.patch
>
>
> Carl Austin raised a good issue in a comment on my Lucene 4.0.0 alpha blog 
> post: 
> http://blog.mikemccandless.com/2012/07/lucene-400-alpha-at-long-last.html
> IndexWriter will now (as of 4.0) delete all foreign files from the index 
> directory.  We made this change because Codecs are free to write to any files 
> now, so the space of filenames is hard to "bound".
> But if the user accidentally uses the wrong directory (eg c:/) then we will 
> in fact delete important stuff.
> I think we can at least use some simple criteria (must start with _, maybe 
> must fit certain pattern eg _(_X).Y), so we are much less likely to 
> delete a non-Lucene file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4190) IndexWriter deletes non-Lucene files

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407181#comment-13407181
 ] 

Uwe Schindler commented on LUCENE-4190:
---

String.startsWith("_") static string is cheap (a few cpu cycles, as it only 
needs compare length and one char... Please dont tell me that SI.addFilkes is 
called in inner loops like Scorers! Not doing this check is stupid.

BTW: In CFSDirectory the assert about double entries on reading the dir should 
also throw CorruptIndexEx, because a CFS with duplicate file names is broken. 
This check is even cheaper. I am planning to open a new issue to fix all those 
I/O related checks to be hard, asserts are not appropriate here.

> IndexWriter deletes non-Lucene files
> 
>
> Key: LUCENE-4190
> URL: https://issues.apache.org/jira/browse/LUCENE-4190
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Robert Muir
> Fix For: 4.0, 5.0
>
> Attachments: LUCENE-4190.patch, LUCENE-4190.patch
>
>
> Carl Austin raised a good issue in a comment on my Lucene 4.0.0 alpha blog 
> post: 
> http://blog.mikemccandless.com/2012/07/lucene-400-alpha-at-long-last.html
> IndexWriter will now (as of 4.0) delete all foreign files from the index 
> directory.  We made this change because Codecs are free to write to any files 
> now, so the space of filenames is hard to "bound".
> But if the user accidentally uses the wrong directory (eg c:/) then we will 
> in fact delete important stuff.
> I think we can at least use some simple criteria (must start with _, maybe 
> must fit certain pattern eg _(_X).Y), so we are much less likely to 
> delete a non-Lucene file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-4190) IndexWriter deletes non-Lucene files

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407177#comment-13407177
 ] 

Robert Muir edited comment on LUCENE-4190 at 7/5/12 2:42 PM:
-

I'm not 1000% determined for it to only be an assert, but then we should change 
how the code works to make
sure that the check is not too expensive. The current assert makes 
SegmentInfo.addFiles/addFile very expensive (if its turned directly into a hard 
check)


  was (Author: rcmuir):
I'm not 1000% determined for it to only be an assert, but then we should 
change how the code works to make
sure that the check is not too expensive. The current assert makes 
SegmentInfo.addFiles/addFile very expensive.

  
> IndexWriter deletes non-Lucene files
> 
>
> Key: LUCENE-4190
> URL: https://issues.apache.org/jira/browse/LUCENE-4190
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Robert Muir
> Fix For: 4.0, 5.0
>
> Attachments: LUCENE-4190.patch, LUCENE-4190.patch
>
>
> Carl Austin raised a good issue in a comment on my Lucene 4.0.0 alpha blog 
> post: 
> http://blog.mikemccandless.com/2012/07/lucene-400-alpha-at-long-last.html
> IndexWriter will now (as of 4.0) delete all foreign files from the index 
> directory.  We made this change because Codecs are free to write to any files 
> now, so the space of filenames is hard to "bound".
> But if the user accidentally uses the wrong directory (eg c:/) then we will 
> in fact delete important stuff.
> I think we can at least use some simple criteria (must start with _, maybe 
> must fit certain pattern eg _(_X).Y), so we are much less likely to 
> delete a non-Lucene file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4190) IndexWriter deletes non-Lucene files

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407177#comment-13407177
 ] 

Robert Muir commented on LUCENE-4190:
-

I'm not 1000% determined for it to only be an assert, but then we should change 
how the code works to make
sure that the check is not too expensive. The current assert makes 
SegmentInfo.addFiles/addFile very expensive.


> IndexWriter deletes non-Lucene files
> 
>
> Key: LUCENE-4190
> URL: https://issues.apache.org/jira/browse/LUCENE-4190
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Robert Muir
> Fix For: 4.0, 5.0
>
> Attachments: LUCENE-4190.patch, LUCENE-4190.patch
>
>
> Carl Austin raised a good issue in a comment on my Lucene 4.0.0 alpha blog 
> post: 
> http://blog.mikemccandless.com/2012/07/lucene-400-alpha-at-long-last.html
> IndexWriter will now (as of 4.0) delete all foreign files from the index 
> directory.  We made this change because Codecs are free to write to any files 
> now, so the space of filenames is hard to "bound".
> But if the user accidentally uses the wrong directory (eg c:/) then we will 
> in fact delete important stuff.
> I think we can at least use some simple criteria (must start with _, maybe 
> must fit certain pattern eg _(_X).Y), so we are much less likely to 
> delete a non-Lucene file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4190) IndexWriter deletes non-Lucene files

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407170#comment-13407170
 ] 

Uwe Schindler commented on LUCENE-4190:
---

{quote}
bq. Assert makes no sense here as it does not prevent people from doing the 
wrong thing.

I don't agree: i at first thought to do a hard check, but this is only really 
necessary for codec developers. So an assert
is enough, because you catch it when developing your codec (its either gonna 
work, or completely not work here).
{quote}

Why not make it a hard check, otherwise one could write a file without _ and 
schwupps, it's wech :) (German). Why only an assert? If we require all files 
start with _ lets enorce it, otherwise delete all files like we do currently. 
Using an assert would get my -1 to commit this again.

> IndexWriter deletes non-Lucene files
> 
>
> Key: LUCENE-4190
> URL: https://issues.apache.org/jira/browse/LUCENE-4190
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Robert Muir
> Fix For: 4.0, 5.0
>
> Attachments: LUCENE-4190.patch, LUCENE-4190.patch
>
>
> Carl Austin raised a good issue in a comment on my Lucene 4.0.0 alpha blog 
> post: 
> http://blog.mikemccandless.com/2012/07/lucene-400-alpha-at-long-last.html
> IndexWriter will now (as of 4.0) delete all foreign files from the index 
> directory.  We made this change because Codecs are free to write to any files 
> now, so the space of filenames is hard to "bound".
> But if the user accidentally uses the wrong directory (eg c:/) then we will 
> in fact delete important stuff.
> I think we can at least use some simple criteria (must start with _, maybe 
> must fit certain pattern eg _(_X).Y), so we are much less likely to 
> delete a non-Lucene file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4190) IndexWriter deletes non-Lucene files

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407167#comment-13407167
 ] 

Robert Muir commented on LUCENE-4190:
-

just to mention: the reason I don't like the Directory refactoring would be 
some of the crazy things we do (look at CompoundFileDirectory and also 
IndexWriter copySegmentAsIs, etc).

This is basically what i think we should avoid: adding a lot of risky 
complexity for little gain.

> IndexWriter deletes non-Lucene files
> 
>
> Key: LUCENE-4190
> URL: https://issues.apache.org/jira/browse/LUCENE-4190
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Robert Muir
> Fix For: 4.0, 5.0
>
> Attachments: LUCENE-4190.patch, LUCENE-4190.patch
>
>
> Carl Austin raised a good issue in a comment on my Lucene 4.0.0 alpha blog 
> post: 
> http://blog.mikemccandless.com/2012/07/lucene-400-alpha-at-long-last.html
> IndexWriter will now (as of 4.0) delete all foreign files from the index 
> directory.  We made this change because Codecs are free to write to any files 
> now, so the space of filenames is hard to "bound".
> But if the user accidentally uses the wrong directory (eg c:/) then we will 
> in fact delete important stuff.
> I think we can at least use some simple criteria (must start with _, maybe 
> must fit certain pattern eg _(_X).Y), so we are much less likely to 
> delete a non-Lucene file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4190) IndexWriter deletes non-Lucene files

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407162#comment-13407162
 ] 

Robert Muir commented on LUCENE-4190:
-

{quote}
I was thinking about the whole thing for longer time. My idea would limit us a 
bit more, but I really like Mike's proposal of fixed names. I would change the 
Directory class, so every method that handles or deletes files gets 2 
parameters, segment name and one arbitrary codec-private file name. the 
directory is then responsible to create the file name, prefix with _ and so on. 
A custom directoy (like hbase), could use the segment name as table name and 
the private file name as identifier, so all segment files go into same hbase 
table. the diurectory would then also be responible to do a "cleanup"/"list of 
files", where it would only return files matching the pattern.
{quote}

I'm not sure matching _[0-9a-z_]+ is really that big of an improvement over 
just the underscore. But i dont think we need
to refactor Directory.java to do this. we could just change the underscore 
check to a regular expression.

{quote}
Assert makes no sense here as it does not prevent people from doing the wrong 
thing.
{quote}

I don't agree: i at first thought to do a hard check, but this is only really 
necessary for codec developers. So an assert
is enough, because you catch it when developing your codec (its either gonna 
work, or completely not work here).

{quote}
If we create an new index, we enforce that listFiles returns empty list (., .. 
excluded, buts thats done already), otherwise we throw IOException("directory 
not empty").
{quote}

I thought about this but i have concerns about things like .DS_Store and 
.nfsX or other files that some system could
be doing behind the scenes, etc.




> IndexWriter deletes non-Lucene files
> 
>
> Key: LUCENE-4190
> URL: https://issues.apache.org/jira/browse/LUCENE-4190
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Robert Muir
> Fix For: 4.0, 5.0
>
> Attachments: LUCENE-4190.patch, LUCENE-4190.patch
>
>
> Carl Austin raised a good issue in a comment on my Lucene 4.0.0 alpha blog 
> post: 
> http://blog.mikemccandless.com/2012/07/lucene-400-alpha-at-long-last.html
> IndexWriter will now (as of 4.0) delete all foreign files from the index 
> directory.  We made this change because Codecs are free to write to any files 
> now, so the space of filenames is hard to "bound".
> But if the user accidentally uses the wrong directory (eg c:/) then we will 
> in fact delete important stuff.
> I think we can at least use some simple criteria (must start with _, maybe 
> must fit certain pattern eg _(_X).Y), so we are much less likely to 
> delete a non-Lucene file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4190) IndexWriter deletes non-Lucene files

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407155#comment-13407155
 ] 

Robert Muir commented on LUCENE-4190:
-

{quote}
So if there are no objections, I'll re-commit the underscore fix (which there 
was consensus for), and then discussion can continue about better methods.
{quote}

I don't object to the patch being committed (though i think it would be good to 
wait a little bit), but
I am still very very concerned that it starts a slippery slope back to 
Codec.files().


> IndexWriter deletes non-Lucene files
> 
>
> Key: LUCENE-4190
> URL: https://issues.apache.org/jira/browse/LUCENE-4190
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Robert Muir
> Fix For: 4.0, 5.0
>
> Attachments: LUCENE-4190.patch, LUCENE-4190.patch
>
>
> Carl Austin raised a good issue in a comment on my Lucene 4.0.0 alpha blog 
> post: 
> http://blog.mikemccandless.com/2012/07/lucene-400-alpha-at-long-last.html
> IndexWriter will now (as of 4.0) delete all foreign files from the index 
> directory.  We made this change because Codecs are free to write to any files 
> now, so the space of filenames is hard to "bound".
> But if the user accidentally uses the wrong directory (eg c:/) then we will 
> in fact delete important stuff.
> I think we can at least use some simple criteria (must start with _, maybe 
> must fit certain pattern eg _(_X).Y), so we are much less likely to 
> delete a non-Lucene file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4190) IndexWriter deletes non-Lucene files

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407157#comment-13407157
 ] 

Uwe Schindler commented on LUCENE-4190:
---

Hi,
I was thinking about the whole thing for longer time. My idea would limit us a 
bit more, but I really like Mike's proposal of fixed names. I would change the 
Directory class, so every method that handles or deletes files gets 2 
parameters, segment name and one arbitrary codec-private file name. the 
directory is then responsible to create the file name, prefix with _ and so on. 
A custom directoy (like hbase), could use the segment name as table name and 
the private file name as identifier, so all segment files go into same hbase 
table. the diurectory would then also be responible to do a "cleanup"/"list of 
files", where it would only return files matching the pattern.

For the index wide metdata like segments file we would then unfortunately need 
a special method to get indexoutput :(

If we keep with current one-filename, i would make the format fixed, so it 
throws IOException if filename is invalid. Assert makes no sense here as it 
does not prevent people from doing the wrong thing. Then really nothing can 
create invalid files and deleting by "_[0-9a-z_]+" works and all would be happy.

Alternatively, we could switch to the following:
- If we create an *new* index, we enforce that listFiles returns empty list (., 
.. excluded, buts thats done already), otherwise we throw 
IOException("directory not empty").
- If there is a segment file already there, we can delete everything not 
allowed in an index.

Uwe

> IndexWriter deletes non-Lucene files
> 
>
> Key: LUCENE-4190
> URL: https://issues.apache.org/jira/browse/LUCENE-4190
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Robert Muir
> Fix For: 4.0, 5.0
>
> Attachments: LUCENE-4190.patch, LUCENE-4190.patch
>
>
> Carl Austin raised a good issue in a comment on my Lucene 4.0.0 alpha blog 
> post: 
> http://blog.mikemccandless.com/2012/07/lucene-400-alpha-at-long-last.html
> IndexWriter will now (as of 4.0) delete all foreign files from the index 
> directory.  We made this change because Codecs are free to write to any files 
> now, so the space of filenames is hard to "bound".
> But if the user accidentally uses the wrong directory (eg c:/) then we will 
> in fact delete important stuff.
> I think we can at least use some simple criteria (must start with _, maybe 
> must fit certain pattern eg _(_X).Y), so we are much less likely to 
> delete a non-Lucene file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4190) IndexWriter deletes non-Lucene files

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407151#comment-13407151
 ] 

Robert Muir commented on LUCENE-4190:
-

{quote}
I didn't see anyone saying that deleting all files was preferable in the short 
term.
{quote}

I'm not sure this is totally true.

Again I think if its required that we have a *perfect* solution, then deleting 
all files is preferable to the alternative of codec having hairy code to detect 
if it owns or doesnt own a file.

But this patch is a nice *imperfect* solution that probably prevents accidental 
deletion of MyImportantDocument.doc or whatever.


> IndexWriter deletes non-Lucene files
> 
>
> Key: LUCENE-4190
> URL: https://issues.apache.org/jira/browse/LUCENE-4190
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Robert Muir
> Fix For: 4.0, 5.0
>
> Attachments: LUCENE-4190.patch, LUCENE-4190.patch
>
>
> Carl Austin raised a good issue in a comment on my Lucene 4.0.0 alpha blog 
> post: 
> http://blog.mikemccandless.com/2012/07/lucene-400-alpha-at-long-last.html
> IndexWriter will now (as of 4.0) delete all foreign files from the index 
> directory.  We made this change because Codecs are free to write to any files 
> now, so the space of filenames is hard to "bound".
> But if the user accidentally uses the wrong directory (eg c:/) then we will 
> in fact delete important stuff.
> I think we can at least use some simple criteria (must start with _, maybe 
> must fit certain pattern eg _(_X).Y), so we are much less likely to 
> delete a non-Lucene file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4190) IndexWriter deletes non-Lucene files

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407148#comment-13407148
 ] 

Yonik Seeley commented on LUCENE-4190:
--

bq. But I think this was premature, because a lot of questions and comments 
came afterwards.

Except that if you read back through the issue, that's not what happened.

Anyway, if you're interested in consensus now, I don't see anyone opposed to 
the underscore solution in the short term, even if some thought it didn't go 
far enough.  I didn't see anyone saying that deleting all files was preferable 
in the short term.
So if there are no objections, I'll re-commit the underscore fix (which there 
was consensus for), and then discussion can continue about better methods.

Robert, I'll repeat your own words back to you:
bq. We can maybe improve in the future besides the _ check, but I just think 
this is an easy improvement that will prevent most of the problems.

> IndexWriter deletes non-Lucene files
> 
>
> Key: LUCENE-4190
> URL: https://issues.apache.org/jira/browse/LUCENE-4190
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Robert Muir
> Fix For: 4.0, 5.0
>
> Attachments: LUCENE-4190.patch, LUCENE-4190.patch
>
>
> Carl Austin raised a good issue in a comment on my Lucene 4.0.0 alpha blog 
> post: 
> http://blog.mikemccandless.com/2012/07/lucene-400-alpha-at-long-last.html
> IndexWriter will now (as of 4.0) delete all foreign files from the index 
> directory.  We made this change because Codecs are free to write to any files 
> now, so the space of filenames is hard to "bound".
> But if the user accidentally uses the wrong directory (eg c:/) then we will 
> in fact delete important stuff.
> I think we can at least use some simple criteria (must start with _, maybe 
> must fit certain pattern eg _(_X).Y), so we are much less likely to 
> delete a non-Lucene file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4190) IndexWriter deletes non-Lucene files

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407145#comment-13407145
 ] 

Robert Muir commented on LUCENE-4190:
-

{quote}
The words, the threat, the quick action - correct - that's my problem.
{quote}

Right, i take offense to the idea that if i committed something too soon, i 
cant back it out.

Sure, it didnt help that I was already frustrated with the technical situation 
(I thought and still do think,
that the patch is a great compromise, easy solution, low risk, simple, etc).

But i think if this situation happens, e.g. someone commits prematurely, then 
there are a bunch of comments
on the issue that make it clear there really isnt consensus, then they have the 
right to back it out, in fact
I think its the right thing to do. 

And nobody can veto that.


> IndexWriter deletes non-Lucene files
> 
>
> Key: LUCENE-4190
> URL: https://issues.apache.org/jira/browse/LUCENE-4190
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Robert Muir
> Fix For: 4.0, 5.0
>
> Attachments: LUCENE-4190.patch, LUCENE-4190.patch
>
>
> Carl Austin raised a good issue in a comment on my Lucene 4.0.0 alpha blog 
> post: 
> http://blog.mikemccandless.com/2012/07/lucene-400-alpha-at-long-last.html
> IndexWriter will now (as of 4.0) delete all foreign files from the index 
> directory.  We made this change because Codecs are free to write to any files 
> now, so the space of filenames is hard to "bound".
> But if the user accidentally uses the wrong directory (eg c:/) then we will 
> in fact delete important stuff.
> I think we can at least use some simple criteria (must start with _, maybe 
> must fit certain pattern eg _(_X).Y), so we are much less likely to 
> delete a non-Lucene file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-4195) Add javadocs to Codec package.html

[Robert Muir (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Muir resolved LUCENE-4195.
-

   Resolution: Fixed
Fix Version/s: 4.0

I committed this, Thanks again!

> Add javadocs to Codec package.html
> --
>
> Key: LUCENE-4195
> URL: https://issues.apache.org/jira/browse/LUCENE-4195
> Project: Lucene - Java
>  Issue Type: Improvement
>  Components: core/codecs
>Affects Versions: 5.0
>Reporter: Alan Woodward
>Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-4195.patch
>
>
> The Codec package.html is pretty basic.  Add some overview information.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4190) IndexWriter deletes non-Lucene files

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407120#comment-13407120
 ] 

Mark Miller commented on LUCENE-4190:
-

bq. You just don't like the words I used.

The words, the threat, the quick action - correct - that's my problem.

Your objective is certainly not my problem.

Had you made the argument you formulated after, I would not have a problem. 
That argument is within the 'sandbox'.

> IndexWriter deletes non-Lucene files
> 
>
> Key: LUCENE-4190
> URL: https://issues.apache.org/jira/browse/LUCENE-4190
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Robert Muir
> Fix For: 4.0, 5.0
>
> Attachments: LUCENE-4190.patch, LUCENE-4190.patch
>
>
> Carl Austin raised a good issue in a comment on my Lucene 4.0.0 alpha blog 
> post: 
> http://blog.mikemccandless.com/2012/07/lucene-400-alpha-at-long-last.html
> IndexWriter will now (as of 4.0) delete all foreign files from the index 
> directory.  We made this change because Codecs are free to write to any files 
> now, so the space of filenames is hard to "bound".
> But if the user accidentally uses the wrong directory (eg c:/) then we will 
> in fact delete important stuff.
> I think we can at least use some simple criteria (must start with _, maybe 
> must fit certain pattern eg _(_X).Y), so we are much less likely to 
> delete a non-Lucene file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4195) Add javadocs to Codec package.html

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407117#comment-13407117
 ] 

Robert Muir commented on LUCENE-4195:
-

awesome! Thanks for doing this.

> Add javadocs to Codec package.html
> --
>
> Key: LUCENE-4195
> URL: https://issues.apache.org/jira/browse/LUCENE-4195
> Project: Lucene - Java
>  Issue Type: Improvement
>  Components: core/codecs
>Affects Versions: 5.0
>Reporter: Alan Woodward
>Priority: Minor
> Attachments: LUCENE-4195.patch
>
>
> The Codec package.html is pretty basic.  Add some overview information.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4195) Add javadocs to Codec package.html

[Alan Woodward (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward updated LUCENE-4195:
--

Attachment: LUCENE-4195.patch

Here's some basic javadoc, telling users how to register new Codecs and 
PostingsFormats.  Pretty basic, but better than nothing!

> Add javadocs to Codec package.html
> --
>
> Key: LUCENE-4195
> URL: https://issues.apache.org/jira/browse/LUCENE-4195
> Project: Lucene - Java
>  Issue Type: Improvement
>  Components: core/codecs
>Affects Versions: 5.0
>Reporter: Alan Woodward
>Priority: Minor
> Attachments: LUCENE-4195.patch
>
>
> The Codec package.html is pretty basic.  Add some overview information.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4190) IndexWriter deletes non-Lucene files

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407115#comment-13407115
 ] 

Mark Miller commented on LUCENE-4190:
-

{quote}Another way to look at it: I committed the patch after Mike reviewed it, 
because it looked like consensus.
There was then a ton of questions and commentary, arguably there wasnt really 
consensus and i prematurely committed. So i backed it out. {quote}

That would have been a good argument and much better than the alternative 
argument you took IMO.

> IndexWriter deletes non-Lucene files
> 
>
> Key: LUCENE-4190
> URL: https://issues.apache.org/jira/browse/LUCENE-4190
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Robert Muir
> Fix For: 4.0, 5.0
>
> Attachments: LUCENE-4190.patch, LUCENE-4190.patch
>
>
> Carl Austin raised a good issue in a comment on my Lucene 4.0.0 alpha blog 
> post: 
> http://blog.mikemccandless.com/2012/07/lucene-400-alpha-at-long-last.html
> IndexWriter will now (as of 4.0) delete all foreign files from the index 
> directory.  We made this change because Codecs are free to write to any files 
> now, so the space of filenames is hard to "bound".
> But if the user accidentally uses the wrong directory (eg c:/) then we will 
> in fact delete important stuff.
> I think we can at least use some simple criteria (must start with _, maybe 
> must fit certain pattern eg _(_X).Y), so we are much less likely to 
> delete a non-Lucene file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org