Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-10 Thread Grant Ingersoll
On Dec 10, 2009, at 1:01 AM, Mattmann, Chris A (388J) wrote: Hi Yonik et al., I¹d like to add: Option C: Sub fields are specified as a attribute on the fieldType tag // needed to essentially define the point

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-10 Thread Mattmann, Chris A (388J)
Hi Grant, On 12/10/09 3:16 AM, Grant Ingersoll gsing...@apache.org wrote: I'm not sure this works, as you need to specify the type of the subfield, which is what Option B does. I don't think inheritance is the what is going on here, more like delegation, and that isn't necessarily needed

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-10 Thread Mattmann, Chris A (388J)
I¹ll try and hold off, but also work on a patch for option (B+)C :) On 12/10/09 7:37 AM, Grant Ingersoll gsing...@apache.org wrote: I have Option B implemented at this point, minus a few tests passing. I'll put up a patch as soon as I get it working.

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-10 Thread Chris Hostetter
: My current thought on #1 is that we probably don't want to change the : internal lookup mechanism used by IndexSchema unless we gain : significant power by doing so. I'm not sure I currently see it. : : My thoughts on #2 is more on a case-by-case basis. For the simple : case of a point class

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-09 Thread Chris Hostetter
: I'm not really understanding the value of an approach like that. for : starters, what Lucene field names would ultimately be created in those : examples? : : The first field would be named location__location. : The second field would be named location_home_location_home. : The third

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-09 Thread Chris Hostetter
: fieldType name=point type=solr.PointType dimension=2 subFieldType=double/ : field name=home type=point indexed=true stored=true/ ... : And a new document of: : doc : field name=point39.0 -79.434/field : /doc : : There are three fields created: : home -- Contains the stored value :

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-09 Thread Chris Hostetter
: That's not how the Cartesian Field stuff works, but I think I see what : you are getting at and I would say I'm going to explicitly punt on that : right now. Ultimately, I think when such a case comes up, the FieldType : needs to be configured to be able to determine this information. I'm

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-09 Thread Grant Ingersoll
On Dec 9, 2009, at 2:04 PM, Chris Hostetter wrote: : fieldType name=point type=solr.PointType dimension=2 subFieldType=double/ : field name=home type=point indexed=true stored=true/ ... : And a new document of: : doc : field name=point39.0 -79.434/field : /doc : : There are

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-09 Thread Yonik Seeley
I haven't followed this whole thread... but I wanted to point out that it probably intersects with the review of grant's latest patch that I did here: https://issues.apache.org/jira/browse/SOLR-1131 I did want to cut'n'paste something from that post: : I do want to separate these two issues

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-09 Thread Mattmann, Chris A (388J)
Hi All, : fieldType name=point type=solr.PointType dimension=2 subFieldType=double/ : field name=home type=point indexed=true stored=true/ ... : And a new document of: : doc : field name=point39.0 -79.434/field : /doc : : There are three fields created: : home -- Contains the

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-09 Thread Yonik Seeley
On Wed, Dec 9, 2009 at 2:41 PM, Yonik Seeley yo...@lucidimagination.com wrote: So... the question is, do we have a concrete alternative to this that is well fleshed out? I do, I do... just a little variant that is geo specific and hence results in nicer names :-) fieldType name=point

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-09 Thread Grant Ingersoll
On Dec 9, 2009, at 2:46 PM, Yonik Seeley wrote: On Wed, Dec 9, 2009 at 2:41 PM, Yonik Seeley yo...@lucidimagination.com wrote: So... the question is, do we have a concrete alternative to this that is well fleshed out? I do, I do... just a little variant that is geo specific and hence

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-09 Thread Yonik Seeley
On Wed, Dec 9, 2009 at 3:21 PM, Grant Ingersoll gsing...@apache.org wrote: Additionally, how do you deal w/ a point in a 3D (or n-D) space? I guess you would go back to the way you did it (0,1,etc). This was really just a naming variation, not really a different approach. I just don't see why

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-09 Thread Grant Ingersoll
On Dec 9, 2009, at 3:47 PM, Yonik Seeley wrote: I thought I defined it well... hmmm. I'll take another stab, outlining using dynamic fields in both scenarios (explicitly defined dynamic fields, and automatically defined as part of the creation of the point class). I think we really do

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-09 Thread Yonik Seeley
On Wed, Dec 9, 2009 at 3:49 PM, Grant Ingersoll gsing...@apache.org wrote: On Dec 9, 2009, at 3:47 PM, Yonik Seeley wrote: I thought I defined it well... hmmm. I'll take another stab, outlining using dynamic fields in both scenarios (explicitly defined dynamic fields, and automatically

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-09 Thread Grant Ingersoll
On Dec 9, 2009, at 3:52 PM, Yonik Seeley wrote: On Wed, Dec 9, 2009 at 3:49 PM, Grant Ingersoll gsing...@apache.org wrote: On Dec 9, 2009, at 3:47 PM, Yonik Seeley wrote: I thought I defined it well... hmmm. I'll take another stab, outlining using dynamic fields in both scenarios

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-09 Thread Yonik Seeley
Proposal for handling points using only the field lookup mechanisms currently in place in IndexSchema: Option A: dynamic fields used for subfields, those dynamic fields need to be explicitly defined in the XML // needed

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-09 Thread Grant Ingersoll
OK, I'm fine w/ taking this type of approach, as opposed to the lookup mechanism I have. Of the two laid out below, there are pros and cons to both, as I see it. I'm inclined towards Option B. This keeps it hidden from the user, but doesn't require extra work for Solr. Let me code it up.

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-09 Thread Mattmann, Chris A (388J)
Hi Yonik et al., I¹d like to add: Option C: Sub fields are specified as a attribute on the fieldType tag // needed to essentially define the point type fieldType name=latlon class=GeoPoint subFieldSuffix=_latlon ../

RE: SOLR-1131 - Multiple Fields per Field Type

2009-12-07 Thread Chris Hostetter
: fieldType name=latlon type=LatLonFieldType pattern=location__* / : fieldType name=latlon_home type=LatLonFieldType pattern=location_home_*/ : fieldType name=latlon_work type=LatLonFieldType pattern=location_home_*/ : : field name=location type=latlon/ : field name=location_home

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-07 Thread Chris Hostetter
: I'm not sure if you worry about it. But I'd argue it isn't natural : anyway. You would do the following instead, which is how any address : book I've ever seen works: : field name=home type=LatLonFT/ : field name=work type=LatLonFT/ ...the home vs work distinction was arbitrary. the point

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-07 Thread Grant Ingersoll
On Dec 7, 2009, at 5:59 PM, Chris Hostetter wrote: : fieldType name=latlon type=LatLonFieldType pattern=location__* / : fieldType name=latlon_home type=LatLonFieldType pattern=location_home_*/ : fieldType name=latlon_work type=LatLonFieldType pattern=location_home_*/ : : field

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-07 Thread Grant Ingersoll
On Dec 7, 2009, at 6:13 PM, Chris Hostetter wrote: : I'm not sure if you worry about it. But I'd argue it isn't natural : anyway. You would do the following instead, which is how any address : book I've ever seen works: : field name=home type=LatLonFT/ : field name=work type=LatLonFT/

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-07 Thread Mattmann, Chris A (388J)
Hi Hoss, : fieldType name=latlon type=LatLonFieldType pattern=location__* / : fieldType name=latlon_home type=LatLonFieldType pattern=location_home_*/ : fieldType name=latlon_work type=LatLonFieldType pattern=location_home_*/ : : field name=location type=latlon/ : field

RE: SOLR-1131 - Multiple Fields per Field Type

2009-12-04 Thread Steven A Rowe
Hi Grant, On 12/02/2009 at 2:30 PM, Grant Ingersoll wrote: I've been noodling around with the idea with the notion of a layered field where variants of a primary token are stored at sub positions of the primary token (instead of in separate copy fields) The Indri search engine (now part of

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-04 Thread Erik Hatcher
And this is also an approach Yonik drafted here for user/tagging design: http://wiki.apache.org/solr/UserTagDesign Erik On Dec 4, 2009, at 1:35 PM, Steven A Rowe wrote: Hi Grant, On 12/02/2009 at 2:30 PM, Grant Ingersoll wrote: I've been noodling around with the idea with the

Re: SOLR-1131 - Multiple Fields per Field Type

2009-12-02 Thread Grant Ingersoll
On Dec 1, 2009, at 1:42 AM, Chris Hostetter wrote: It feels like something we've overlooked in this discussion is whether we need to worry about any FieldType API changes needed to make these new PolyField classes aware of when they are multivalued. The API suggestions grant made gives

Re: SOLR-1131 - Multiple Fields per Field Type

2009-11-30 Thread Chris Hostetter
: Maybe, but something needs that logic. Think relational database -- if you : try and add a field to a schema (e.g., using some DBMS client GUI or vanilla : command line SQL) where that name already exists, then you get a SQL : exception. Similarly, SOLR should support such concepts. Maybe it

Re: SOLR-1131 - Multiple Fields per Field Type

2009-11-30 Thread Chris Hostetter
It feels like something we've overlooked in this discussion is whether we need to worry about any FieldType API changes needed to make these new PolyField classes aware of when they are multivalued. The API suggestions grant made gives the FieldTYpe the ability to return a Filed[] from a

RE: SOLR-1131 - Multiple Fields per Field Type

2009-11-30 Thread Mattmann, Chris A (388J)
Hey Hoss, So rather then try to make it entirely magical and behind the scnes, and still require them to know about it if a collision happens and they get an error, let's put it right out in front of them so they know about it and think it through. +1 to that -- was never trying to make

RE: SOLR-1131 - Multiple Fields per Field Type

2009-11-30 Thread Mattmann, Chris A (388J)
Hey Hoss, From: Chris Hostetter [hossman_luc...@fucit.org] Sent: Monday, November 30, 2009 5:42 PM To: solr-dev@lucene.apache.org Subject: Re: SOLR-1131 - Multiple Fields per Field Type It feels like something we've overlooked in this discussion is whether

Re: SOLR-1131 - Multiple Fields per Field Type

2009-11-29 Thread Grant Ingersoll
On Nov 28, 2009, at 7:37 PM, Chris Hostetter wrote: : I don't think it's useful to somehow programmatically access the list : of fields that a fieldType could output. based on my understanding of the potential types of use cases we're talking about, i think i agree with you. It seems

Re: SOLR-1131 - Multiple Fields per Field Type

2009-11-29 Thread Chris Hostetter
: One thing that concerns me is potential field name collision -- where one : of these new multifield producing FieldTypes might want to creat a name : that happens to collide with a field the user has already declared. : : Since FieldTypes are provided an instance of

Re: SOLR-1131 - Multiple Fields per Field Type

2009-11-29 Thread Chris Hostetter
: I thought about this too. It is what Local Solr currently does : (although it expects a certain prefix, too, I believe). However, it : seems a bit unnecessary, as now the user needs to use both the field : type and the dynamic field in order to get it to work, whereas I don't : think they

Re: SOLR-1131 - Multiple Fields per Field Type

2009-11-29 Thread Mattmann, Chris A (388J)
Hi Hoss, On 11/29/09 12:22 PM, Chris Hostetter hossman_luc...@fucit.org wrote: that would tell them if a field name is currently in use, but not what to do about it if it is already in use -- FieldType classes shouldn't need complicated hueristics to figure out somethign the user could

Re: SOLR-1131 - Multiple Fields per Field Type

2009-11-28 Thread Erik Hatcher
What about rather than conflating field types for creating multiple fields, use update processors to do the this expansion instead? Erik On Nov 26, 2009, at 10:04 AM, Grant Ingersoll wrote: On Nov 25, 2009, at 8:24 PM, Chris Hostetter wrote: I'm having a hard time wrapping my

Re: SOLR-1131 - Multiple Fields per Field Type

2009-11-28 Thread Grant Ingersoll
On Nov 28, 2009, at 3:45 AM, Erik Hatcher wrote: What about rather than conflating field types for creating multiple fields, use update processors to do the this expansion instead? How do you maintain the semantic information needed at search time? Are you still having the field type (or

Re: SOLR-1131 - Multiple Fields per Field Type

2009-11-28 Thread Chris Male
Hi, Aren't search semantics the responsibility of a Query Parser and Querys themselves? Just as the semantics of boolean queries are handled by the standard Query parsers and BooleanQuery. On Sat, Nov 28, 2009 at 3:17 PM, Grant Ingersoll gsing...@apache.orgwrote: On Nov 28, 2009, at 3:45 AM,

Re: SOLR-1131 - Multiple Fields per Field Type

2009-11-28 Thread Yonik Seeley
On Sat, Nov 28, 2009 at 9:41 AM, Chris Male gento...@gmail.com wrote: Aren't search semantics the responsibility of a Query Parser and Querys themselves?  Just as the semantics of boolean queries are handled by the standard Query parsers and BooleanQuery. At a certain point, one needs

Re: SOLR-1131 - Multiple Fields per Field Type

2009-11-28 Thread Chris Male
Hi, There is some standardization of the syntax and semantics of range queries, function queries and sorting that exists outside of the field types themselves though. For example for range queries FieldType expects there is just 2 values that define the range I think. Thats a requirement that

Re: SOLR-1131 - Multiple Fields per Field Type

2009-11-28 Thread Yonik Seeley
On Sat, Nov 28, 2009 at 10:51 AM, Chris Male gento...@gmail.com wrote: By allowing each FieldType to have its own search semantics We're far enough removed from an actual feature, I'm not sure if we're disagreeing about anything concrete :-) Going back to Grant's original question, I think it's

Re: SOLR-1131 - Multiple Fields per Field Type

2009-11-28 Thread Chris Hostetter
: I don't think it's useful to somehow programmatically access the list : of fields that a fieldType could output. based on my understanding of the potential types of use cases we're talking about, i think i agree with you. It seems like the most crucial aspect is that a FieldType has a way

Re: SOLR-1131 - Multiple Fields per Field Type

2009-11-28 Thread Mattmann, Chris A (388J)
Hey Hoss, On 11/28/09 4:37 PM, Chris Hostetter hossman_luc...@fucit.org wrote: One thing that concerns me is potential field name collision -- where one of these new multifield producing FieldTypes might want to creat a name that happens to collide with a field the user has already declared.

Re: SOLR-1131 - Multiple Fields per Field Type

2009-11-28 Thread Yonik Seeley
On Sat, Nov 28, 2009 at 7:37 PM, Chris Hostetter hossman_luc...@fucit.org wrote: Using Double underscores kind of feels like a hack, what i keep wondering is if we can't leverage dynamicFields here. This is what the prototype patch does I just put up on SOLR-1131. I've gone with option A for

Re: SOLR-1131 - Multiple Fields per Field Type

2009-11-26 Thread Grant Ingersoll
On Nov 25, 2009, at 8:24 PM, Chris Hostetter wrote: I'm having a hard time wrapping my head arround this entire concept ... i know part of my problem is that your example use case seems somewhat nonsensical... : As a simple proof of concept, imagine that I define a new FieldType :

Re: SOLR-1131 - Multiple Fields per Field Type

2009-11-25 Thread Chris Hostetter
I'm having a hard time wrapping my head arround this entire concept ... i know part of my problem is that your example use case seems somewhat nonsensical... : As a simple proof of concept, imagine that I define a new FieldType : called PlusMinusIntFieldType that extends IntField. This