Why do we need SolrPluginUtils#optimizePreFetchDocs()

[Noble Paul നോബിള്‍ नोब्ळ्]

This looks like a hack. It currently only uses highlighter for
prefetching docs and fields . There is no standard way of other
components to take part in this.

We should either remove this altogether or have a standard way for all
components to take part in this.

-- 
-
Noble Paul | Systems Architect| AOL | http://aol.com

[jira] Commented: (SOLR-1612) Use java.sql.Array to populate Multivalue field in DataImportHandler

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796530#action_12796530
 ] 

Noble Paul commented on SOLR-1612:
--

SOLR-1675 is a different issue. We can change the description of this issue and 
commit the transformer as a part of this

> Use java.sql.Array to populate Multivalue field in DataImportHandler
> 
>
> Key: SOLR-1612
> URL: https://issues.apache.org/jira/browse/SOLR-1612
> Project: Solr
>  Issue Type: Improvement
>  Components: contrib - DataImportHandler
>Affects Versions: 1.4
>Reporter: Matt Inger
> Fix For: 1.5
>
> Attachments: SOLR-1612.patch
>
>
> I am attaching a patch to the JdbcDataSource class which allows the it to 
> populate a multiValued field from a query which returns a java.sql.Array type 
> for a column value.
> In order to make this work, you need to do the following:
> 1.  In your schema.xml file, set the "multiValued" attribute to the value 
> "true"
>  
> 2.  Have your sql query return a sql type of java.sql.Types.ARRAY.  How to do 
> this is very database dependent, but in oracle, you first create a type:
>  create or replace type FOO_TYPE IS TABLE OF VARCHAR2(255);
>   and then when you select the values, you have to use MULTISET and cast 
> to the created type (assuming here that FOO_PARENT_ID is the condition to
>   limit which foos belong to the current record):
>  CAST(MULTISET(SELECT FOO FROM FOOS WHERE 
> FOO_PARENT=FOO_PARENT_ID) AS FOO_TYPE ) MY_FOOS
> The main advantage this has is it can save you a ton of queries over the 
> current method which will fire off an individual query for each document in 
> order to retrieve the values for each multiValued field.  This method 
> executes much faster, and saves alot of database resources.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1682) Implement CollapseComponent

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796529#action_12796529
 ] 

Noble Paul commented on SOLR-1682:
--



bq. But now also a search is done twice. Shouldn't it be better to have two 
complete distinct collectors that don't depend on one another?

Both the collectors are designed to complement each other so that one can 
piggyback on other and minimize the code/work

bq. The field group.numFound is then added as well. Why did you gave it a 
different name?

The names are up for debate . Let us reach a consensus on that . When 
collapse.threshhold=1 collapse.cout/collapse.groupSize can be avoided

> Implement CollapseComponent
> ---
>
> Key: SOLR-1682
> URL: https://issues.apache.org/jira/browse/SOLR-1682
> Project: Solr
>  Issue Type: Sub-task
>  Components: search
>Reporter: Martijn van Groningen
>Assignee: Shalin Shekhar Mangar
> Fix For: 1.5
>
> Attachments: field-collapsing.patch, SOLR-1682.patch, SOLR-236.patch
>
>
> Child issue of SOLR-236. This issue is dedicated to field collapsing in 
> general and all its code (CollapseComponent, DocumentCollapsers and 
> CollapseCollectors). The main goal is the finalize the request parameters and 
> response format.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1568) Implement Spatial Filter

[patrick o'leary (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796524#action_12796524
 ] 

patrick o'leary commented on SOLR-1568:
---

bq. A smart spatial filter, at a minimum, only does the distance calculation 
for those documents that lie within the bounding box. That's the main point to 
the bounding box. I had assumed this is what Patrick's local-solr already did?

It's a little more contrived - I actually executed the lucene query as part of 
the filter, 
* Bounded box
* Text query standard query 
* And bitset
* Pass in that bitset to radial filter to only calculate distances of both text 
search, and bounded box

Thus only doc's matching the query within the bounding box have their distances 
calculated.
Check out 
DistanceQueryBuilder

{code}
public Filter getFilter(Query query) {
// Chain the Query (as filter) with our distance filter
if (distanceFilter != null) {
  distanceFilter.reset();
}
QueryWrapperFilter qf = new QueryWrapperFilter(query);
return new ChainedFilter(new Filter[] {qf, filter},
 ChainedFilter.AND);
  }

public Query getQuery(Query query){
return new ConstantScoreQuery(getFilter(query));
  }
{code}



> Implement Spatial Filter
> 
>
> Key: SOLR-1568
> URL: https://issues.apache.org/jira/browse/SOLR-1568
> Project: Solr
>  Issue Type: New Feature
>Reporter: Grant Ingersoll
>Assignee: Grant Ingersoll
>Priority: Minor
> Fix For: 1.5
>
> Attachments: CartesianTierQParserPlugin.java
>
>
> Given an index with spatial information (either as a geohash, 
> SpatialTileField (see SOLR-1586) or just two lat/lon pairs), we should be 
> able to pass in a filter query that takes in the field name, lat, lon and 
> distance and produces an appropriate Filter (i.e. one that is aware of the 
> underlying field type for use by Solr. 
> The interface _could_ look like:
> {code}
> &fq={!sfilt dist=20}location:49.32,-79.0
> {code}
> or it could be:
> {code}
> &fq={!sfilt lat=49.32 lat=-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt p=49.32,-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt lat=49.32,-79.0 fl=lat,lon dist=20}
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1568) Implement Spatial Filter

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796491#action_12796491
 ] 

Yonik Seeley commented on SOLR-1568:


bq. For those who only want pure radial search, it is the only way, scalable or 
not, unless I'm missing something.

A smart spatial filter, at a minimum, only does the distance calculation for 
those documents that lie within the bounding box.  That's the main point to the 
bounding box.  I had assumed this is what Patrick's local-solr already did?
Although I do think it's a good idea to also allow a user to specify *only* a 
bounding box if they don't want to pay the price for the distance calculation 
at all.

bq. 2. Range query (i.e. bounding box)

And how would that be specified out of the box? (example please?)  You're not 
suggesting that be delegated to the user do you?  That's very hard, very field 
specific, and will often result in multiple range queries, not one.

> Implement Spatial Filter
> 
>
> Key: SOLR-1568
> URL: https://issues.apache.org/jira/browse/SOLR-1568
> Project: Solr
>  Issue Type: New Feature
>Reporter: Grant Ingersoll
>Assignee: Grant Ingersoll
>Priority: Minor
> Fix For: 1.5
>
> Attachments: CartesianTierQParserPlugin.java
>
>
> Given an index with spatial information (either as a geohash, 
> SpatialTileField (see SOLR-1586) or just two lat/lon pairs), we should be 
> able to pass in a filter query that takes in the field name, lat, lon and 
> distance and produces an appropriate Filter (i.e. one that is aware of the 
> underlying field type for use by Solr. 
> The interface _could_ look like:
> {code}
> &fq={!sfilt dist=20}location:49.32,-79.0
> {code}
> or it could be:
> {code}
> &fq={!sfilt lat=49.32 lat=-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt p=49.32,-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt lat=49.32,-79.0 fl=lat,lon dist=20}
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1568) Implement Spatial Filter

[Grant Ingersoll (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796485#action_12796485
 ] 

Grant Ingersoll commented on SOLR-1568:
---

For those who only want pure radial search, it is the only way, scalable or 
not, unless I'm missing something.  

As I see, it there are several options for filtering that already work out of 
the box and that someone may want to use:
1. Radial distance as above
2. Range query (i.e. bounding box)

The one remaining that isn't implemented is the tile stuff.  

Now, obviously, we can provide tools to make these slightly easier to consume.  
I've already prototyped most of it, but in doing so, I just couldn't shake the 
sense that it is a waste of time for anything but the tile stuff and that it 
hides too much in terms of what the tradeoffs are going to be.  I just think it 
is easy enough for an app designer to use their existing toolbox (#1 and #2) 
while throwing in one more lightweight, easy to use new tool (tile support.)

> Implement Spatial Filter
> 
>
> Key: SOLR-1568
> URL: https://issues.apache.org/jira/browse/SOLR-1568
> Project: Solr
>  Issue Type: New Feature
>Reporter: Grant Ingersoll
>Assignee: Grant Ingersoll
>Priority: Minor
> Fix For: 1.5
>
> Attachments: CartesianTierQParserPlugin.java
>
>
> Given an index with spatial information (either as a geohash, 
> SpatialTileField (see SOLR-1586) or just two lat/lon pairs), we should be 
> able to pass in a filter query that takes in the field name, lat, lon and 
> distance and produces an appropriate Filter (i.e. one that is aware of the 
> underlying field type for use by Solr. 
> The interface _could_ look like:
> {code}
> &fq={!sfilt dist=20}location:49.32,-79.0
> {code}
> or it could be:
> {code}
> &fq={!sfilt lat=49.32 lat=-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt p=49.32,-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt lat=49.32,-79.0 fl=lat,lon dist=20}
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1568) Implement Spatial Filter

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796478#action_12796478
 ] 

Yonik Seeley commented on SOLR-1568:


While it's nice that this works (to double-check results, etc), this isn't 
particularly scalable as it calculates the distance (the expensive part) for 
every document.  Or are you suggesting some other kind of implementation in 
conjunction with this syntax?

> Implement Spatial Filter
> 
>
> Key: SOLR-1568
> URL: https://issues.apache.org/jira/browse/SOLR-1568
> Project: Solr
>  Issue Type: New Feature
>Reporter: Grant Ingersoll
>Assignee: Grant Ingersoll
>Priority: Minor
> Fix For: 1.5
>
> Attachments: CartesianTierQParserPlugin.java
>
>
> Given an index with spatial information (either as a geohash, 
> SpatialTileField (see SOLR-1586) or just two lat/lon pairs), we should be 
> able to pass in a filter query that takes in the field name, lat, lon and 
> distance and produces an appropriate Filter (i.e. one that is aware of the 
> underlying field type for use by Solr. 
> The interface _could_ look like:
> {code}
> &fq={!sfilt dist=20}location:49.32,-79.0
> {code}
> or it could be:
> {code}
> &fq={!sfilt lat=49.32 lat=-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt p=49.32,-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt lat=49.32,-79.0 fl=lat,lon dist=20}
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1568) Implement Spatial Filter

[Grant Ingersoll (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796476#action_12796476
 ] 

Grant Ingersoll commented on SOLR-1568:
---

{code}
fq={!frange l=0 u=400}hsin(0.57, -1.3, lat_rad, lon_rad, 3963.205)
{code}



> Implement Spatial Filter
> 
>
> Key: SOLR-1568
> URL: https://issues.apache.org/jira/browse/SOLR-1568
> Project: Solr
>  Issue Type: New Feature
>Reporter: Grant Ingersoll
>Assignee: Grant Ingersoll
>Priority: Minor
> Fix For: 1.5
>
> Attachments: CartesianTierQParserPlugin.java
>
>
> Given an index with spatial information (either as a geohash, 
> SpatialTileField (see SOLR-1586) or just two lat/lon pairs), we should be 
> able to pass in a filter query that takes in the field name, lat, lon and 
> distance and produces an appropriate Filter (i.e. one that is aware of the 
> underlying field type for use by Solr. 
> The interface _could_ look like:
> {code}
> &fq={!sfilt dist=20}location:49.32,-79.0
> {code}
> or it could be:
> {code}
> &fq={!sfilt lat=49.32 lat=-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt p=49.32,-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt lat=49.32,-79.0 fl=lat,lon dist=20}
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1568) Implement Spatial Filter

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796457#action_12796457
 ] 

Yonik Seeley commented on SOLR-1568:


bq. since if we use the FunctionRangeQParser, it's inherent in the passing in 
of the function.

It's not clear exactly what you're proposing - can you give an example request 
so we can compare?

> Implement Spatial Filter
> 
>
> Key: SOLR-1568
> URL: https://issues.apache.org/jira/browse/SOLR-1568
> Project: Solr
>  Issue Type: New Feature
>Reporter: Grant Ingersoll
>Assignee: Grant Ingersoll
>Priority: Minor
> Fix For: 1.5
>
> Attachments: CartesianTierQParserPlugin.java
>
>
> Given an index with spatial information (either as a geohash, 
> SpatialTileField (see SOLR-1586) or just two lat/lon pairs), we should be 
> able to pass in a filter query that takes in the field name, lat, lon and 
> distance and produces an appropriate Filter (i.e. one that is aware of the 
> underlying field type for use by Solr. 
> The interface _could_ look like:
> {code}
> &fq={!sfilt dist=20}location:49.32,-79.0
> {code}
> or it could be:
> {code}
> &fq={!sfilt lat=49.32 lat=-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt p=49.32,-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt lat=49.32,-79.0 fl=lat,lon dist=20}
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-1682) Implement CollapseComponent

[Martijn van Groningen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-1682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Martijn van Groningen updated SOLR-1682:


Attachment: SOLR-1682.patch

Shalin, I tried your patch out and I ran into a few problems with sorting and 
the collapse counts which turned out to be bugs.
# When I was sorting in ascending order(on a field or score), the order was 
itself was incorrect.
# The collapse count was always one (when threshold=1 which is default was 
specified). I suppose the count should increment every time a document is 
collapsed.

I fixed these issues in the new patch and added tests that show that.

Though I have a question about the response format. When collapse.threshold is 
> 1 and more than one documents is collapsed then the collapse.count is named 
group.size. The field group.numFound is then added as well. Why did you gave it 
a different name?

When collapse.threshold is larger than one two collectors are used. I 
understand that in both situations a different algorithm is used. But now also 
a search is done twice. Shouldn't it be better to have two complete distinct 
collectors that don't depend on one another?

> Implement CollapseComponent
> ---
>
> Key: SOLR-1682
> URL: https://issues.apache.org/jira/browse/SOLR-1682
> Project: Solr
>  Issue Type: Sub-task
>  Components: search
>Reporter: Martijn van Groningen
>Assignee: Shalin Shekhar Mangar
> Fix For: 1.5
>
> Attachments: field-collapsing.patch, SOLR-1682.patch, SOLR-236.patch
>
>
> Child issue of SOLR-236. This issue is dedicated to field collapsing in 
> general and all its code (CollapseComponent, DocumentCollapsers and 
> CollapseCollectors). The main goal is the finalize the request parameters and 
> response format.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Re: svn commit: r895750 - /lucene/solr/trunk/src/java/org/apache/solr/search/function/distance/DistanceUtils.java

[Grant Ingersoll]

On Jan 4, 2010, at 5:30 PM, Yonik Seeley wrote:

> On Mon, Jan 4, 2010 at 5:07 PM, Grant Ingersoll  wrote:
>> 
>> On Jan 4, 2010, at 4:19 PM, Yonik Seeley wrote:
>> 
>>> On Mon, Jan 4, 2010 at 2:29 PM,   wrote:
 +  public static final double KM_TO_MILES = 0.621371192;
 +  public static final double MILES_TO_KM = 1.609344;
>>> 
>>> I don't care if these exist, but what are your plans for actually using 
>>> them?
>> 
>> Probably premature to commit on my part, I was working on SOLR-1568 and was 
>> allowing the user to pass in the units for the distance value.
> 
> I still think it's no simpler for a client, and more complex over all.
> You either must require units to be passed in (yuck) or decide on
> default units.  Once you have decided on default units, extra
> parameters for different units is just increased complexity that is
> just as trivial for the client to implement.  They either have to know
> the code for what units they are using or they have to know how to
> convert to the standard units - about the same amount of complexity.
> 
>>> For spatial search, it seems like we should simply standardize on
>>> something, probably either meters or kilometers and be done with it.
>>> It's trivial for clients to convert (and clients aren't end-users),
>>> and will reduce confusion about how to specify units, etc.
>>> 
>>> Likewise for points/locations - they should simply be lat,lon in
>>> degrees.  No need to specify if it's in radians or degrees when
>>> degrees is more of an external standard and it's as simple for a
>>> client to convert as it is to specify.
>> 
>> Possibly, except you can save a few operations per document if you just 
>> store radians when using haversine.
> 
> A single multiply (~3cycles?).  If that's worth saving, we should just
> index it that way for the user...

Sure, point type could have an init parameter, I suppose, that specified 
whether to convert.  Or, the user can just send it in radians to begin with.  
What I want as a designer is to specify it up front based on the type of 
accuracy I want out of my distances.  To me, that's what it all comes back to.  
The app designer doing spatial says:  how accurate do I need my calculations to 
be?  Then, they make decisions about data structures based on that.

> but given the computational cost of
> haversine, it's really in the noise... we should figure out other ways
> to speed things up.
> 

Times 20-100 million records to score/filter?  Not a huge amount of savings, 
but still could be worthwhile for some applications under high load and w/ lots 
of docs without costing anyone else anything different. 


> A location in the xml, when using our built-in field types should be
> unambiguously degrees in lat,lon format.  How it's indexed to increase
> speed, save space, etc, is up to the field type and it's
> configuration.

Actually, it is unambiguous as x,y(,z...).  We have points in a n-dimensional 
space, as of now, but we can add lat/lon specifically if that helps.

> 
>> I'm just not sure I see this as a big deal.  Technically, we could hide all 
>> the complexity of numerics from the user too, but yet we offer ints, floats 
>> and doubles (we could parse them on our side and figure out which is what).
> 
> But we do hide the complexity of numerics from the user (clients) as
> much as we can.  popularity:10 popularity:[5 TO 10] all work without
> the client knowing what kind of numeric field is being used (with the
> exception of plain numerics which are offered only for compatibility
> with existing lucene indexes).

I just don't get why normal spatial calculations are any different from other 
function queries, with the exception, right now, of tiles.  Perhaps if in the 
future we have other complex types that require one offs, then we can unify on 
hiding all of this, but for now the _only_ thing that doesn't work out of the 
box are tiles.  All the rest can be handled through function queries and the 
FunctionRangeQParser.  I don't see much benefit in writing/maintaining code 
that is very marginally more readable than using function queries and will be 
templated in an application anyway and then left to do its job.

-Grant

[jira] Commented: (SOLR-1698) load balanced distributed search

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796417#action_12796417
 ] 

Yonik Seeley commented on SOLR-1698:


Picking which shard replica to request can be random (round-robin or whatever, 
and customizable in the future), but a single distributed request should use 
the same replica for all phases of the request when possible.


> load balanced distributed search
> 
>
> Key: SOLR-1698
> URL: https://issues.apache.org/jira/browse/SOLR-1698
> Project: Solr
>  Issue Type: Improvement
>Reporter: Yonik Seeley
>
> Provide syntax and implementation of load-balancing across shard replicas.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1698) load balanced distributed search

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796410#action_12796410
 ] 

Yonik Seeley commented on SOLR-1698:


This is related to the solr cloud branch, but is perhaps separable enough that 
I thought I'd try integrating into trunk.
We could add pipe delimiters between equivalent shards - useful for testing, 
troubleshooting, and ad hoc requests, even when shard info will be retrieved 
from zookeeper.

Example shards param:
shards=localhost:8983/solr|localhost:8985/solr,localhost:7574/solr|localhost:7576/solr

> load balanced distributed search
> 
>
> Key: SOLR-1698
> URL: https://issues.apache.org/jira/browse/SOLR-1698
> Project: Solr
>  Issue Type: Improvement
>Reporter: Yonik Seeley
>
> Provide syntax and implementation of load-balancing across shard replicas.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (SOLR-1698) load balanced distributed search

[Yonik Seeley (JIRA)]

load balanced distributed search


 Key: SOLR-1698
 URL: https://issues.apache.org/jira/browse/SOLR-1698
 Project: Solr
  Issue Type: Improvement
Reporter: Yonik Seeley


Provide syntax and implementation of load-balancing across shard replicas.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Re: svn commit: r895750 - /lucene/solr/trunk/src/java/org/apache/solr/search/function/distance/DistanceUtils.java

[Yonik Seeley]

On Mon, Jan 4, 2010 at 5:07 PM, Grant Ingersoll  wrote:
>
> On Jan 4, 2010, at 4:19 PM, Yonik Seeley wrote:
>
>> On Mon, Jan 4, 2010 at 2:29 PM,   wrote:
>>> +  public static final double KM_TO_MILES = 0.621371192;
>>> +  public static final double MILES_TO_KM = 1.609344;
>>
>> I don't care if these exist, but what are your plans for actually using them?
>
> Probably premature to commit on my part, I was working on SOLR-1568 and was 
> allowing the user to pass in the units for the distance value.

I still think it's no simpler for a client, and more complex over all.
You either must require units to be passed in (yuck) or decide on
default units.  Once you have decided on default units, extra
parameters for different units is just increased complexity that is
just as trivial for the client to implement.  They either have to know
the code for what units they are using or they have to know how to
convert to the standard units - about the same amount of complexity.

>> For spatial search, it seems like we should simply standardize on
>> something, probably either meters or kilometers and be done with it.
>> It's trivial for clients to convert (and clients aren't end-users),
>> and will reduce confusion about how to specify units, etc.
>>
>> Likewise for points/locations - they should simply be lat,lon in
>> degrees.  No need to specify if it's in radians or degrees when
>> degrees is more of an external standard and it's as simple for a
>> client to convert as it is to specify.
>
> Possibly, except you can save a few operations per document if you just store 
> radians when using haversine.

A single multiply (~3cycles?).  If that's worth saving, we should just
index it that way for the user... but given the computational cost of
haversine, it's really in the noise... we should figure out other ways
to speed things up.

A location in the xml, when using our built-in field types should be
unambiguously degrees in lat,lon format.  How it's indexed to increase
speed, save space, etc, is up to the field type and it's
configuration.

> I'm just not sure I see this as a big deal.  Technically, we could hide all 
> the complexity of numerics from the user too, but yet we offer ints, floats 
> and doubles (we could parse them on our side and figure out which is what).

But we do hide the complexity of numerics from the user (clients) as
much as we can.  popularity:10 popularity:[5 TO 10] all work without
the client knowing what kind of numeric field is being used (with the
exception of plain numerics which are offered only for compatibility
with existing lucene indexes).

>  I'm more of the mindset that I think the app designer should be able to make 
> the choice, but possibly with some guidance from us as to what is appropriate 
> for each situation, just as we do with other field types.

Yes, absolutely.  The app *designer* can make the choices and use the
appropriate field types and config, and we should isolate clients from
those choices (and changes in those choices) to the degree that it's
practical.  That's what we currently do.

-Yonik
http://www.lucidimagination.com

Re: svn commit: r895750 - /lucene/solr/trunk/src/java/org/apache/solr/search/function/distance/DistanceUtils.java

[Grant Ingersoll]

On Jan 4, 2010, at 4:19 PM, Yonik Seeley wrote:

> On Mon, Jan 4, 2010 at 2:29 PM,   wrote:
>> +  public static final double KM_TO_MILES = 0.621371192;
>> +  public static final double MILES_TO_KM = 1.609344;
> 
> I don't care if these exist, but what are your plans for actually using them?

Probably premature to commit on my part, I was working on SOLR-1568 and was 
allowing the user to pass in the units for the distance value.

> 
> For spatial search, it seems like we should simply standardize on
> something, probably either meters or kilometers and be done with it.
> It's trivial for clients to convert (and clients aren't end-users),
> and will reduce confusion about how to specify units, etc.
> 
> Likewise for points/locations - they should simply be lat,lon in
> degrees.  No need to specify if it's in radians or degrees when
> degrees is more of an external standard and it's as simple for a
> client to convert as it is to specify.

Possibly, except you can save a few operations per document if you just store 
radians when using haversine. 

I'm just not sure I see this as a big deal.  Technically, we could hide all the 
complexity of numerics from the user too, but yet we offer ints, floats and 
doubles (we could parse them on our side and figure out which is what).  I'm 
more of the mindset that I think the app designer should be able to make the 
choice, but possibly with some guidance from us as to what is appropriate for 
each situation, just as we do with other field types.

Re: svn commit: r895750 - /lucene/solr/trunk/src/java/org/apache/solr/search/function/distance/DistanceUtils.java

[Yonik Seeley]

On Mon, Jan 4, 2010 at 2:29 PM,   wrote:
> +  public static final double KM_TO_MILES = 0.621371192;
> +  public static final double MILES_TO_KM = 1.609344;

I don't care if these exist, but what are your plans for actually using them?

For spatial search, it seems like we should simply standardize on
something, probably either meters or kilometers and be done with it.
It's trivial for clients to convert (and clients aren't end-users),
and will reduce confusion about how to specify units, etc.

Likewise for points/locations - they should simply be lat,lon in
degrees.  No need to specify if it's in radians or degrees when
degrees is more of an external standard and it's as simple for a
client to convert as it is to specify.

-Yonik
http://www.lucidimagination.com

[jira] Commented: (SOLR-1568) Implement Spatial Filter

[Ryan McKinley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796352#action_12796352
 ] 

Ryan McKinley commented on SOLR-1568:
-

{quote}
could be written as 
{code}
&fq={!sfilt f=location}
&p=49,-77
&d=1000
{code}
{quote}

Jeez -- I had no idea...  then I think (most?) cases are taken care of...

With this, a handler could be defined with the default param: 'fq={!sfilt 
f=location}' and the 'end user' could simply call:
  &p=49,-77&d=1000
never needing to grock the local-params/function syntax.


> Implement Spatial Filter
> 
>
> Key: SOLR-1568
> URL: https://issues.apache.org/jira/browse/SOLR-1568
> Project: Solr
>  Issue Type: New Feature
>Reporter: Grant Ingersoll
>Assignee: Grant Ingersoll
>Priority: Minor
> Fix For: 1.5
>
> Attachments: CartesianTierQParserPlugin.java
>
>
> Given an index with spatial information (either as a geohash, 
> SpatialTileField (see SOLR-1586) or just two lat/lon pairs), we should be 
> able to pass in a filter query that takes in the field name, lat, lon and 
> distance and produces an appropriate Filter (i.e. one that is aware of the 
> underlying field type for use by Solr. 
> The interface _could_ look like:
> {code}
> &fq={!sfilt dist=20}location:49.32,-79.0
> {code}
> or it could be:
> {code}
> &fq={!sfilt lat=49.32 lat=-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt p=49.32,-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt lat=49.32,-79.0 fl=lat,lon dist=20}
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1568) Implement Spatial Filter

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796348#action_12796348
 ] 

Yonik Seeley commented on SOLR-1568:


bq. Also, in all of these proposals, how do we intend to specify what distance 
calculation to use?

IMO, the user shouldn't have to.  The vast majority of users want "normal" 
spatial search... fast yet good enough.   They won't know the details and won't 
want to know (except perhaps that it's accurate to x meters).

For those super-advanced users that wish to specify their own distance 
function, we could allow an optional parameter on sfilt that specifies an 
arbitrary function.

> Implement Spatial Filter
> 
>
> Key: SOLR-1568
> URL: https://issues.apache.org/jira/browse/SOLR-1568
> Project: Solr
>  Issue Type: New Feature
>Reporter: Grant Ingersoll
>Assignee: Grant Ingersoll
>Priority: Minor
> Fix For: 1.5
>
> Attachments: CartesianTierQParserPlugin.java
>
>
> Given an index with spatial information (either as a geohash, 
> SpatialTileField (see SOLR-1586) or just two lat/lon pairs), we should be 
> able to pass in a filter query that takes in the field name, lat, lon and 
> distance and produces an appropriate Filter (i.e. one that is aware of the 
> underlying field type for use by Solr. 
> The interface _could_ look like:
> {code}
> &fq={!sfilt dist=20}location:49.32,-79.0
> {code}
> or it could be:
> {code}
> &fq={!sfilt lat=49.32 lat=-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt p=49.32,-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt lat=49.32,-79.0 fl=lat,lon dist=20}
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1568) Implement Spatial Filter

[Grant Ingersoll (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796308#action_12796308
 ] 

Grant Ingersoll commented on SOLR-1568:
---

Also, in all of these proposals, how do we intend to specify what distance 
calculation to use?  Which is another reason for not going down the path of a 
generic QParser for this, since if we use the FunctionRangeQParser, it's 
inherent in the passing in of the function.

> Implement Spatial Filter
> 
>
> Key: SOLR-1568
> URL: https://issues.apache.org/jira/browse/SOLR-1568
> Project: Solr
>  Issue Type: New Feature
>Reporter: Grant Ingersoll
>Assignee: Grant Ingersoll
>Priority: Minor
> Fix For: 1.5
>
> Attachments: CartesianTierQParserPlugin.java
>
>
> Given an index with spatial information (either as a geohash, 
> SpatialTileField (see SOLR-1586) or just two lat/lon pairs), we should be 
> able to pass in a filter query that takes in the field name, lat, lon and 
> distance and produces an appropriate Filter (i.e. one that is aware of the 
> underlying field type for use by Solr. 
> The interface _could_ look like:
> {code}
> &fq={!sfilt dist=20}location:49.32,-79.0
> {code}
> or it could be:
> {code}
> &fq={!sfilt lat=49.32 lat=-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt p=49.32,-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt lat=49.32,-79.0 fl=lat,lon dist=20}
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1568) Implement Spatial Filter

[Grant Ingersoll (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796299#action_12796299
 ] 

Grant Ingersoll commented on SOLR-1568:
---

bq. Are we filtering by bounding box or by distance, sorting by distance, or 
boosting by distance? 

I see this as a filtering only issue.  To me, all the other capabilities other 
than filtering by a Spatial Tile Field are already covered through normal Solr 
capabilities, so I don't see much benefit to inventing new syntax and more code 
to maintain even if it is slightly easier for the app designer to consume.

> Implement Spatial Filter
> 
>
> Key: SOLR-1568
> URL: https://issues.apache.org/jira/browse/SOLR-1568
> Project: Solr
>  Issue Type: New Feature
>Reporter: Grant Ingersoll
>Assignee: Grant Ingersoll
>Priority: Minor
> Fix For: 1.5
>
> Attachments: CartesianTierQParserPlugin.java
>
>
> Given an index with spatial information (either as a geohash, 
> SpatialTileField (see SOLR-1586) or just two lat/lon pairs), we should be 
> able to pass in a filter query that takes in the field name, lat, lon and 
> distance and produces an appropriate Filter (i.e. one that is aware of the 
> underlying field type for use by Solr. 
> The interface _could_ look like:
> {code}
> &fq={!sfilt dist=20}location:49.32,-79.0
> {code}
> or it could be:
> {code}
> &fq={!sfilt lat=49.32 lat=-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt p=49.32,-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt lat=49.32,-79.0 fl=lat,lon dist=20}
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1602) Refactor SOLR package structure to include o.a.solr.response and move QueryResponseWriters in there

[Andrzej Bialecki (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796291#action_12796291
 ] 

Andrzej Bialecki  commented on SOLR-1602:
-

I'm in favor of B. This worked well in Hadoop (mapred -> mapreduce) where the 
list of deprecations was massive and API changes were not straightforward at 
all - still it was done to promote a better design and allow new functionality. 
Whole deprecated hierarchies live there for at least two major releases, and 
surely they were visible to thousands of Hadoop devs. The downside was 
occasional confusion, and of course the porting effort required to use the new 
API, but the upside was an excellent back-compat to keep serious users happy, 
and a clear message to all to get prepared for the switch.

So IMHO having a bunch of deprecated classes for a while is not a big deal, if 
it gives us freedom to pursue a better design.

> Refactor SOLR package structure to include o.a.solr.response and move 
> QueryResponseWriters in there
> ---
>
> Key: SOLR-1602
> URL: https://issues.apache.org/jira/browse/SOLR-1602
> Project: Solr
>  Issue Type: Improvement
>  Components: Response Writers
>Affects Versions: 1.2, 1.3, 1.4
> Environment: independent of environment (code structure)
>Reporter: Chris A. Mattmann
>Assignee: Noble Paul
> Fix For: 1.5
>
> Attachments: SOLR-1602.Mattmann.112509.patch.txt, 
> SOLR-1602.Mattmann.112509_02.patch.txt, upgrade_solr_config
>
>
> Currently all o.a.solr.request.QueryResponseWriter implementations are 
> curiously located in the o.a.solr.request package. Not only is this package 
> getting big (30+ classes), a lot of them are misplaced. There should be a 
> first-class o.a.solr.response package, and the response related classes 
> should be given a home there. Patch forthcoming.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1568) Implement Spatial Filter

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796289#action_12796289
 ] 

Yonik Seeley commented on SOLR-1568:


The examples are not of equivalent power.  What field are we working off of?  
Are we filtering by bounding box or by distance, sorting by distance, or 
boosting by distance?

Further, given that query parsers default to grabbing global params if local 
ones aren't specified this
{code}
&fq={!sfilt p=49,-77 d=1000 f=location}
{code}

could be written as 
{code}
&fq={!sfilt f=location}
&p=49,-77
&d=1000
{code}

Which I assumed people would do anyway to make it easier for clients to 
construct queries and do filtering + sorting w/o repeating parameters.

> Implement Spatial Filter
> 
>
> Key: SOLR-1568
> URL: https://issues.apache.org/jira/browse/SOLR-1568
> Project: Solr
>  Issue Type: New Feature
>Reporter: Grant Ingersoll
>Assignee: Grant Ingersoll
>Priority: Minor
> Fix For: 1.5
>
> Attachments: CartesianTierQParserPlugin.java
>
>
> Given an index with spatial information (either as a geohash, 
> SpatialTileField (see SOLR-1586) or just two lat/lon pairs), we should be 
> able to pass in a filter query that takes in the field name, lat, lon and 
> distance and produces an appropriate Filter (i.e. one that is aware of the 
> underlying field type for use by Solr. 
> The interface _could_ look like:
> {code}
> &fq={!sfilt dist=20}location:49.32,-79.0
> {code}
> or it could be:
> {code}
> &fq={!sfilt lat=49.32 lat=-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt p=49.32,-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt lat=49.32,-79.0 fl=lat,lon dist=20}
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1602) Refactor SOLR package structure to include o.a.solr.response and move QueryResponseWriters in there

[Ryan McKinley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796285#action_12796285
 ] 

Ryan McKinley commented on SOLR-1602:
-

In an effort to get some resolution here...

There are only three options:
A. Leave it as is
B. Refactor *with* deprecations
--C. Refactor without deprecations--

C is out, so A and B are the only options worth discussing.

The advantage of B is that the package:
 o.a.s.response would be clean and organized
(but o.a.s.request would still have a bunch of deprecated files)

The arguments for A amount to: "things are fine as they are", or "the confusion 
of changing is not worth whatever slight gain we would get"

The strong resistance is to 'C', I suspect wider ambivalence towards 'B'




> Refactor SOLR package structure to include o.a.solr.response and move 
> QueryResponseWriters in there
> ---
>
> Key: SOLR-1602
> URL: https://issues.apache.org/jira/browse/SOLR-1602
> Project: Solr
>  Issue Type: Improvement
>  Components: Response Writers
>Affects Versions: 1.2, 1.3, 1.4
> Environment: independent of environment (code structure)
>Reporter: Chris A. Mattmann
>Assignee: Noble Paul
> Fix For: 1.5
>
> Attachments: SOLR-1602.Mattmann.112509.patch.txt, 
> SOLR-1602.Mattmann.112509_02.patch.txt, upgrade_solr_config
>
>
> Currently all o.a.solr.request.QueryResponseWriter implementations are 
> curiously located in the o.a.solr.request package. Not only is this package 
> getting big (30+ classes), a lot of them are misplaced. There should be a 
> first-class o.a.solr.response package, and the response related classes 
> should be given a home there. Patch forthcoming.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1568) Implement Spatial Filter

[Grant Ingersoll (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796282#action_12796282
 ] 

Grant Ingersoll commented on SOLR-1568:
---

Keep in mind distance functions are about more than just lat/lon, although 
there is no reason why we can't support both.

> Implement Spatial Filter
> 
>
> Key: SOLR-1568
> URL: https://issues.apache.org/jira/browse/SOLR-1568
> Project: Solr
>  Issue Type: New Feature
>Reporter: Grant Ingersoll
>Assignee: Grant Ingersoll
>Priority: Minor
> Fix For: 1.5
>
> Attachments: CartesianTierQParserPlugin.java
>
>
> Given an index with spatial information (either as a geohash, 
> SpatialTileField (see SOLR-1586) or just two lat/lon pairs), we should be 
> able to pass in a filter query that takes in the field name, lat, lon and 
> distance and produces an appropriate Filter (i.e. one that is aware of the 
> underlying field type for use by Solr. 
> The interface _could_ look like:
> {code}
> &fq={!sfilt dist=20}location:49.32,-79.0
> {code}
> or it could be:
> {code}
> &fq={!sfilt lat=49.32 lat=-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt p=49.32,-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt lat=49.32,-79.0 fl=lat,lon dist=20}
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1568) Implement Spatial Filter

[Ryan McKinley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796280#action_12796280
 ] 

Ryan McKinley commented on SOLR-1568:
-

I'm torn on this also...

The function query syntax (and all local params really) are a bit cryptic.  
When I look into it though they are really powerful, and i see an advantage to 
having that level of control. (and it works in solr now)

I wonder if there is a simple way to add query rewriting that would convert:
 &lat=49&long=-77&radius=10
to:
 &fq={!sfilt p=49,-77 f=location dist=10}

Perhaps a RequestRewrite component?  

I doubt adding something like http://tuckey.org/urlrewrite/ would work 
off-the-shelf, but it could go a long way to hide the ugly innards for a simple 
case like this, yet still allow the power of the function query syntax.


> Implement Spatial Filter
> 
>
> Key: SOLR-1568
> URL: https://issues.apache.org/jira/browse/SOLR-1568
> Project: Solr
>  Issue Type: New Feature
>Reporter: Grant Ingersoll
>Assignee: Grant Ingersoll
>Priority: Minor
> Fix For: 1.5
>
> Attachments: CartesianTierQParserPlugin.java
>
>
> Given an index with spatial information (either as a geohash, 
> SpatialTileField (see SOLR-1586) or just two lat/lon pairs), we should be 
> able to pass in a filter query that takes in the field name, lat, lon and 
> distance and produces an appropriate Filter (i.e. one that is aware of the 
> underlying field type for use by Solr. 
> The interface _could_ look like:
> {code}
> &fq={!sfilt dist=20}location:49.32,-79.0
> {code}
> or it could be:
> {code}
> &fq={!sfilt lat=49.32 lat=-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt p=49.32,-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt lat=49.32,-79.0 fl=lat,lon dist=20}
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1568) Implement Spatial Filter

[Grant Ingersoll (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796249#action_12796249
 ] 

Grant Ingersoll commented on SOLR-1568:
---

Here's the thing I keep coming back to with this issue.  For all the field 
types other than SpatialTile (Cartesian Tier), filters are already easily 
creatable.  It seems odd to have to write this special QParserPlugin that has 
all this syntax support, etc. to create filters for all of the other cases 
beside the spatial tile.  For instance, pure radial distance filtering is 
easily handled through the FunctionRangeQParserPlugin and the various distance 
functions.  In fact, going this route is more convoluted for those cases b/c 
now you need a way to pass in what distance function you want to use.

> Implement Spatial Filter
> 
>
> Key: SOLR-1568
> URL: https://issues.apache.org/jira/browse/SOLR-1568
> Project: Solr
>  Issue Type: New Feature
>Reporter: Grant Ingersoll
>Assignee: Grant Ingersoll
>Priority: Minor
> Fix For: 1.5
>
> Attachments: CartesianTierQParserPlugin.java
>
>
> Given an index with spatial information (either as a geohash, 
> SpatialTileField (see SOLR-1586) or just two lat/lon pairs), we should be 
> able to pass in a filter query that takes in the field name, lat, lon and 
> distance and produces an appropriate Filter (i.e. one that is aware of the 
> underlying field type for use by Solr. 
> The interface _could_ look like:
> {code}
> &fq={!sfilt dist=20}location:49.32,-79.0
> {code}
> or it could be:
> {code}
> &fq={!sfilt lat=49.32 lat=-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt p=49.32,-79.0 f=location dist=20}
> {code}
> or:
> {code}
> &fq={!sfilt lat=49.32,-79.0 fl=lat,lon dist=20}
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1612) Use java.sql.Array to populate Multivalue field in DataImportHandler

[Matt Inger (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796211#action_12796211
 ] 

Matt Inger commented on SOLR-1612:
--

Yes, and I have attached that transformer to SOLR-1675, and will link to this 
ticket.  Perhaps it's useful enough to include directly in the contrib code.

> Use java.sql.Array to populate Multivalue field in DataImportHandler
> 
>
> Key: SOLR-1612
> URL: https://issues.apache.org/jira/browse/SOLR-1612
> Project: Solr
>  Issue Type: Improvement
>  Components: contrib - DataImportHandler
>Affects Versions: 1.4
>Reporter: Matt Inger
> Fix For: 1.5
>
> Attachments: SOLR-1612.patch
>
>
> I am attaching a patch to the JdbcDataSource class which allows the it to 
> populate a multiValued field from a query which returns a java.sql.Array type 
> for a column value.
> In order to make this work, you need to do the following:
> 1.  In your schema.xml file, set the "multiValued" attribute to the value 
> "true"
>  
> 2.  Have your sql query return a sql type of java.sql.Types.ARRAY.  How to do 
> this is very database dependent, but in oracle, you first create a type:
>  create or replace type FOO_TYPE IS TABLE OF VARCHAR2(255);
>   and then when you select the values, you have to use MULTISET and cast 
> to the created type (assuming here that FOO_PARENT_ID is the condition to
>   limit which foos belong to the current record):
>  CAST(MULTISET(SELECT FOO FROM FOOS WHERE 
> FOO_PARENT=FOO_PARENT_ID) AS FOO_TYPE ) MY_FOOS
> The main advantage this has is it can save you a ton of queries over the 
> current method which will fire off an individual query for each document in 
> order to retrieve the values for each multiValued field.  This method 
> executes much faster, and saves alot of database resources.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1602) Refactor SOLR package structure to include o.a.solr.response and move QueryResponseWriters in there

[Chris A. Mattmann (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796185#action_12796185
 ] 

Chris A. Mattmann commented on SOLR-1602:
-

bq. I bet 99% of the users do not care about a wrongly named package when 
everything works. But they care when things stop working. Code organization is 
secondary to usability. Let us not cause discomfort to our users for such a 
trivial issue.

I wouldn't be so quick to bet what 99% of your "users" think without some solid 
evidence to back that up. And no, code organization is not second to usability 
to all parties -- it depends on what hat you're using. To you it may be, but 
not to everyone.

bq. End users do not dictate code level organization but they do have an 
influence when compatibility is involved. In this case, it is an inconvenience 
for many of them which can be avoided easily, so why not?

Because satisfying end-users is only looking at one of the stakeholders of the 
system and ignoring the others.

bq. I agree with Hoss. This is too much discussion over too small an issue. I 
think things are quite clear. Hoss, Erik, Noble and I all feel that breaking 
compatibility is not worth it. So lets do what needs to be done and get on with 
more important things.

The issue is far from clear as far as I'm concerned and I'm the one who 
reported the issue. The comments on this issue from others are evidence of 
that. The issue will be satisfied when there is clear consensus from the 
parties involved.


> Refactor SOLR package structure to include o.a.solr.response and move 
> QueryResponseWriters in there
> ---
>
> Key: SOLR-1602
> URL: https://issues.apache.org/jira/browse/SOLR-1602
> Project: Solr
>  Issue Type: Improvement
>  Components: Response Writers
>Affects Versions: 1.2, 1.3, 1.4
> Environment: independent of environment (code structure)
>Reporter: Chris A. Mattmann
>Assignee: Noble Paul
> Fix For: 1.5
>
> Attachments: SOLR-1602.Mattmann.112509.patch.txt, 
> SOLR-1602.Mattmann.112509_02.patch.txt, upgrade_solr_config
>
>
> Currently all o.a.solr.request.QueryResponseWriter implementations are 
> curiously located in the o.a.solr.request package. Not only is this package 
> getting big (30+ classes), a lot of them are misplaced. There should be a 
> first-class o.a.solr.response package, and the response related classes 
> should be given a home there. Patch forthcoming.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1316) Create autosuggest component

[Brad Giaccio (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796182#action_12796182
 ] 

Brad Giaccio commented on SOLR-1316:


bq. We could do that but as Andrej noted, we'd end up re-implementing a lot of 
its functionality. I'm not sure if it is worth it. I agree that it'd be odd 
using parameters prefixed with "spellcheck" for auto-suggest and it'd have been 
easier if it were vice-versa. Does anybody have a suggestion?

Couldn't you just extend the SpellCheckComponent, and make use of something 
like COMPONENT_NAME or PARAM_PREFIX in the param calls instead of the static 
string in SpellingParams?  That way the autosuggestcomponent would have 
COMPONENT_NAME=autoSuggest and the spelling would have it set to spelling then 
let all the common code just live in the base class.

Just a thought

> Create autosuggest component
> 
>
> Key: SOLR-1316
> URL: https://issues.apache.org/jira/browse/SOLR-1316
> Project: Solr
>  Issue Type: New Feature
>  Components: search
>Affects Versions: 1.4
>Reporter: Jason Rutherglen
>Assignee: Shalin Shekhar Mangar
>Priority: Minor
> Fix For: 1.5
>
> Attachments: suggest.patch, suggest.patch, suggest.patch, TST.zip
>
>   Original Estimate: 96h
>  Remaining Estimate: 96h
>
> Autosuggest is a common search function that can be integrated
> into Solr as a SearchComponent. Our first implementation will
> use the TernaryTree found in Lucene contrib. 
> * Enable creation of the dictionary from the index or via Solr's
> RPC mechanism
> * What types of parameters and settings are desirable?
> * Hopefully in the future we can include user click through
> rates to boost those terms/phrases higher

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Build failed in Hudson: Solr-trunk #1022

[Apache Hudson Server]

See 

--
[...truncated 1860 lines...]
AUclient/ruby/flare/public/images/pie_31.png
AUclient/ruby/flare/public/images/pie_32.png
AUclient/ruby/flare/public/images/pie_33.png
AUclient/ruby/flare/public/images/pie_34.png
AUclient/ruby/flare/public/images/pie_70.png
AUclient/ruby/flare/public/images/pie_35.png
AUclient/ruby/flare/public/images/pie_71.png
AUclient/ruby/flare/public/images/pie_36.png
AUclient/ruby/flare/public/images/pie_72.png
AUclient/ruby/flare/public/images/pie_37.png
AUclient/ruby/flare/public/images/pie_73.png
AUclient/ruby/flare/public/images/pie_38.png
AUclient/ruby/flare/public/images/pie_74.png
AUclient/ruby/flare/public/images/pie_39.png
AUclient/ruby/flare/public/images/pie_75.png
AUclient/ruby/flare/public/images/pie_76.png
AUclient/ruby/flare/public/images/pie_77.png
AUclient/ruby/flare/public/images/pie_78.png
AUclient/ruby/flare/public/images/x-close.gif
AUclient/ruby/flare/public/dispatch.fcgi
A client/ruby/flare/public/500.html
A client/ruby/flare/public/javascripts
A client/ruby/flare/public/javascripts/prototype.js
A client/ruby/flare/public/javascripts/effects.js
A client/ruby/flare/public/javascripts/dragdrop.js
A client/ruby/flare/public/javascripts/application.js
A client/ruby/flare/public/javascripts/controls.js
A client/ruby/flare/public/404.html
A client/ruby/flare/public/.htaccess
A client/ruby/flare/public/stylesheets
A client/ruby/flare/public/stylesheets/flare.css
A client/ruby/flare/public/favicon.ico
A client/ruby/solr-ruby
A client/ruby/solr-ruby/solr
A client/ruby/solr-ruby/solr/conf
AUclient/ruby/solr-ruby/solr/conf/schema.xml
A client/ruby/solr-ruby/solr/conf/protwords.txt
A client/ruby/solr-ruby/solr/conf/stopwords.txt
AUclient/ruby/solr-ruby/solr/conf/solrconfig.xml
A client/ruby/solr-ruby/solr/conf/xslt
A client/ruby/solr-ruby/solr/conf/xslt/example.xsl
A client/ruby/solr-ruby/solr/conf/scripts.conf
A client/ruby/solr-ruby/solr/conf/admin-extra.html
A client/ruby/solr-ruby/solr/conf/synonyms.txt
A client/ruby/solr-ruby/solr/lib
A client/ruby/solr-ruby/test
A client/ruby/solr-ruby/test/unit
A client/ruby/solr-ruby/test/unit/standard_response_test.rb
A client/ruby/solr-ruby/test/unit/document_test.rb
AUclient/ruby/solr-ruby/test/unit/select_test.rb
AUclient/ruby/solr-ruby/test/unit/delimited_file_source_test.rb
A client/ruby/solr-ruby/test/unit/xpath_test_file.xml
AUclient/ruby/solr-ruby/test/unit/array_mapper_test.rb
AUclient/ruby/solr-ruby/test/unit/solr_mock_base.rb
A client/ruby/solr-ruby/test/unit/field_test.rb
AUclient/ruby/solr-ruby/test/unit/modify_document_test.rb
A client/ruby/solr-ruby/test/unit/add_document_test.rb
AUclient/ruby/solr-ruby/test/unit/xpath_mapper_test.rb
AUclient/ruby/solr-ruby/test/unit/request_test.rb
A client/ruby/solr-ruby/test/unit/commit_test.rb
AUclient/ruby/solr-ruby/test/unit/suite.rb
AUclient/ruby/solr-ruby/test/unit/changes_yaml_test.rb
A client/ruby/solr-ruby/test/unit/spellcheck_response_test.rb
A client/ruby/solr-ruby/test/unit/ping_test.rb
A client/ruby/solr-ruby/test/unit/dismax_request_test.rb
AUclient/ruby/solr-ruby/test/unit/indexer_test.rb
A client/ruby/solr-ruby/test/unit/response_test.rb
AUclient/ruby/solr-ruby/test/unit/connection_test.rb
A client/ruby/solr-ruby/test/unit/delete_test.rb
AUclient/ruby/solr-ruby/test/unit/tab_delimited.txt
A client/ruby/solr-ruby/test/unit/hpricot_test_file.xml
AUclient/ruby/solr-ruby/test/unit/standard_request_test.rb
A client/ruby/solr-ruby/test/unit/hpricot_mapper_test.rb
A client/ruby/solr-ruby/test/unit/spellchecker_request_test.rb
AUclient/ruby/solr-ruby/test/unit/data_mapper_test.rb
AUclient/ruby/solr-ruby/test/unit/util_test.rb
A client/ruby/solr-ruby/test/functional
A client/ruby/solr-ruby/test/functional/test_solr_server.rb
A client/ruby/solr-ruby/test/functional/server_test.rb
A client/ruby/solr-ruby/test/conf
AUclient/ruby/solr-ruby/test/conf/schema.xml
A client/ruby/solr-ruby/test/conf/protwords.txt
A client/ruby/solr-ruby/test/conf/stopwords.txt
AUclient/ruby/solr-ruby/test/conf/solrconfig.xml
A client/ruby/solr-ruby/test/conf/scripts.conf
A client/ruby/solr-ruby/test/conf/admin-extra.html
A client/ruby/solr-ruby/test/conf/synonyms.txt
A client/ruby/solr-ruby/LICENSE.txt
A   

[jira] Updated: (SOLR-1697) PluginInfo should load plugins w/o class attribute also

[Noble Paul (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-1697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul updated SOLR-1697:
-

Attachment: SOLR-1697.patch

> PluginInfo should load plugins w/o class attribute also
> ---
>
> Key: SOLR-1697
> URL: https://issues.apache.org/jira/browse/SOLR-1697
> Project: Solr
>  Issue Type: Improvement
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Minor
> Fix For: 1.5
>
> Attachments: SOLR-1697.patch
>
>
> This should enable components to load plugins w/ a default classname too

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Assigned: (SOLR-1697) PluginInfo should load plugins w/o class attribute also

[Noble Paul (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-1697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul reassigned SOLR-1697:


Assignee: Noble Paul

> PluginInfo should load plugins w/o class attribute also
> ---
>
> Key: SOLR-1697
> URL: https://issues.apache.org/jira/browse/SOLR-1697
> Project: Solr
>  Issue Type: Improvement
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Minor
> Fix For: 1.5
>
> Attachments: SOLR-1697.patch
>
>
> This should enable components to load plugins w/ a default classname too

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (SOLR-1697) PluginInfo should load plugins w/o class attribute also

[Noble Paul (JIRA)]

PluginInfo should load plugins w/o class attribute also
---

 Key: SOLR-1697
 URL: https://issues.apache.org/jira/browse/SOLR-1697
 Project: Solr
  Issue Type: Improvement
Reporter: Noble Paul
Priority: Minor
 Fix For: 1.5


This should enable components to load plugins w/ a default classname too

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1696) Deprecate old syntax and move configuration to HighlightComponent

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796167#action_12796167
 ] 

Noble Paul commented on SOLR-1696:
--

The new syntax can be as follows
{code:xml}

  
   
   
   

 100

   

   
   

  
  70
  
  0.5
  
  [-\w ,/\n\"']{20,200}

   

   
   

 
 

   
  

{code}

This way SolrCore can be totally agnostic of highlighter 

> Deprecate old  syntax and move configuration to 
> HighlightComponent
> 
>
> Key: SOLR-1696
> URL: https://issues.apache.org/jira/browse/SOLR-1696
> Project: Solr
>  Issue Type: Improvement
>  Components: highlighter
>Reporter: Noble Paul
> Fix For: 1.5
>
>
> There is no reason why we should have a custom syntax for highlighter 
> configuration.
> It can be treated like any other SearchComponent and all the configuration 
> can go in there.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (SOLR-1696) Deprecate old syntax and move configuration to HighlightComponent

[Noble Paul (JIRA)]

Deprecate old  syntax and move configuration to HighlightComponent


 Key: SOLR-1696
 URL: https://issues.apache.org/jira/browse/SOLR-1696
 Project: Solr
  Issue Type: Improvement
  Components: highlighter
Reporter: Noble Paul
 Fix For: 1.5


There is no reason why we should have a custom syntax for highlighter 
configuration.

It can be treated like any other SearchComponent and all the configuration can 
go in there.


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Re: highlighting setting in solrconfig.xml

[Noble Paul നോബിള്‍ नोब्ळ्]

new issue https://issues.apache.org/jira/browse/SOLR-1696

2010/1/4 noble.paul :
> Koji has a point. the  syntax has to be deprecated . All the 
> configurations can be but into the HighlightComponent.
>
>
> I shall open an issue.
>
> On Mon, Jan 4, 2010 at 9:44 AM, Chris Hostetter  
> wrote:
>>
>> : If this design is ok, I need to introduce new sub tags like
>> :  and  in 
>> : in solrconfig.xml. But now I wonder why the highligher settings
>> : are in such very original place rather than .
>> :
>> : Do we have any reason to keep  tag?
>> : Or can we move it into HighlightComponent?
>>
>> I'm not all that aware of what all is involved in the existing
>>  config options, but i suspect it was introduced *before*
>> search components, as a way to configure the highlighting utils that were
>> used by multiple request handlers ... moving all of that into init/request
>> params for the HighlightingComponent seems like a good idea to me -- but i
>> wouldn't be suprised if there were some things that make sense to leave as
>> independently initialized objects that are then refrenced by name,
>> similar to the way QParserPlugins are initialized seperately from the
>> QueryComponent and then refered to by name at request time.
>>
>> but as i said: i know very little about highlighting.
>>
>>
>>
>> -Hoss
>>
>>
>
>
>
> --
> -
> Noble Paul | Systems Architect| AOL | http://aol.com
>
>



-- 
-
Noble Paul | Systems Architect| AOL | http://aol.com

Re: highlighting setting in solrconfig.xml

[Noble Paul നോബിള്‍ नोब्ळ्]

Koji has a point. the  syntax has to be deprecated . All
the configurations can be but into the HighlightComponent.


I shall open an issue.

On Mon, Jan 4, 2010 at 9:44 AM, Chris Hostetter
 wrote:
>
> : If this design is ok, I need to introduce new sub tags like
> :  and  in 
> : in solrconfig.xml. But now I wonder why the highligher settings
> : are in such very original place rather than .
> :
> : Do we have any reason to keep  tag?
> : Or can we move it into HighlightComponent?
>
> I'm not all that aware of what all is involved in the existing
>  config options, but i suspect it was introduced *before*
> search components, as a way to configure the highlighting utils that were
> used by multiple request handlers ... moving all of that into init/request
> params for the HighlightingComponent seems like a good idea to me -- but i
> wouldn't be suprised if there were some things that make sense to leave as
> independently initialized objects that are then refrenced by name,
> similar to the way QParserPlugins are initialized seperately from the
> QueryComponent and then refered to by name at request time.
>
> but as i said: i know very little about highlighting.
>
>
>
> -Hoss
>
>



-- 
-
Noble Paul | Systems Architect| AOL | http://aol.com

[jira] Updated: (SOLR-1664) Some Methods in FieldType actually should be in SchemaField

[Noble Paul (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-1664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul updated SOLR-1664:
-

Attachment: SOLR-1664.patch

updated to trunk

> Some Methods in FieldType actually should be in SchemaField
> ---
>
> Key: SOLR-1664
> URL: https://issues.apache.org/jira/browse/SOLR-1664
> Project: Solr
>  Issue Type: Improvement
>Reporter: Noble Paul
> Fix For: 1.5
>
> Attachments: SOLR-1664.patch, SOLR-1664.patch
>
>
> The following methods are only overridable from FieldType. 
> {code:java}
> public Field createField(SchemaField field, String externalVal, float boost) ;
> protected Field.TermVector getFieldTermVec(SchemaField field,String 
> internalVal) ;
> protected Field.Store getFieldStore(SchemaField field,String internalVal);
> protected Field.Index getFieldIndex(SchemaField field,String internalVal);
> public ValueSource getValueSource(SchemaField field, QParser parser);
> public Query getRangeQuery(QParser parser, SchemaField field, String part1, 
> String part2, boolean minInclusive, boolean maxInclusive) ;
> {code}
> if there is anything specific to be done on a per-field basis it is not 
> possible. if we allow overriding SchemaField it will be more flexible.
> I hope this can make it simpler to implement SOLR-1131
> So ,there are two ways of overriding a functionality.
> * Override Fieldtype#createSchemaField() and provide new implementation of 
> SchemaField
> * Override methods from FieldType itself(if the behavior is not per field ) 
> as it is now
> SchemaField is made non-final and Solr only invokes methods on SchemaField 
> and not FieldType directly

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1268) Incorporate Lucene's FastVectorHighlighter

[Koji Sekiguchi (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796147#action_12796147
 ] 

Koji Sekiguchi commented on SOLR-1268:
--

I'll commit in a few days if nobody objects.

> Incorporate Lucene's FastVectorHighlighter
> --
>
> Key: SOLR-1268
> URL: https://issues.apache.org/jira/browse/SOLR-1268
> Project: Solr
>  Issue Type: New Feature
>  Components: highlighter
>Reporter: Koji Sekiguchi
>Assignee: Koji Sekiguchi
>Priority: Minor
> Fix For: 1.5
>
> Attachments: SOLR-1268.patch, SOLR-1268.patch
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1677) Add support for o.a.lucene.util.Version for BaseTokenizerFactory and BaseTokenFilterFactory

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796136#action_12796136
 ] 

Robert Muir commented on SOLR-1677:
---

{quote}
User Carl helpfully replies...

That was identified as a bug with FooTokenFilter that was fixed in Lucene 
3.1, but the default behavior was left as is for backcompatibility. If you 
change your  value to 3.1 (or 3.2) you'll get 
the newer/better behavior - but if you used FooTokenFilterFactory in an index 
analyzer you'll need to reindex.
{quote}

User Carl isn't helpful, user Carl is an idiot.

The javadoc of Version in lucene clearly says:
{noformat}
 * WARNING: When changing the version parameter
 * that you supply to components in Lucene, do not simply
 * change the version at search-time, but instead also adjust
 * your indexing code to match, and re-index.
{noformat}

User Carl could also tell Bob that its ok to index with ArabicAnalyzer and 
query with ChineseAnalyzer, this kind of stupid theoretical situation isn't any 
kind of valid logical argument against having a default value for this.


> Add support for o.a.lucene.util.Version for BaseTokenizerFactory and 
> BaseTokenFilterFactory
> ---
>
> Key: SOLR-1677
> URL: https://issues.apache.org/jira/browse/SOLR-1677
> Project: Solr
>  Issue Type: Sub-task
>  Components: Schema and Analysis
>Reporter: Uwe Schindler
> Attachments: SOLR-1677.patch, SOLR-1677.patch, SOLR-1677.patch, 
> SOLR-1677.patch
>
>
> Since Lucene 2.9, a lot of analyzers use a Version constant to keep backwards 
> compatibility with old indexes created using older versions of Lucene. The 
> most important example is StandardTokenizer, which changed its behaviour with 
> posIncr and incorrect host token types in 2.4 and also in 2.9.
> In Lucene 3.0 this matchVersion ctor parameter is mandatory and in 3.1, with 
> much more Unicode support, almost every Tokenizer/TokenFilter needs this 
> Version parameter. In 2.9, the deprecated old ctors without Version take 
> LUCENE_24 as default to mimic the old behaviour, e.g. in StandardTokenizer.
> This patch adds basic support for the Lucene Version property to the base 
> factories. Subclasses then can use the luceneMatchVersion decoded enum (in 
> 3.0) / Parameter (in 2.9) for constructing Tokenstreams. The code currently 
> contains a helper map to decode the version strings, but in 3.0 is can be 
> replaced by Version.valueOf(String), as the Version is a subclass of Java5 
> enums. The default value is Version.LUCENE_24 (as this is the default for the 
> no-version ctors in Lucene).
> This patch also removes unneeded conversions to CharArraySet from 
> StopFilterFactory (now done by Lucene since 2.9). The generics are also fixed 
> to match Lucene 3.0.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1612) Use java.sql.Array to populate Multivalue field in DataImportHandler

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796128#action_12796128
 ] 

Noble Paul commented on SOLR-1612:
--

is your issue resolved? Is it not possible to make it a Transformer?

> Use java.sql.Array to populate Multivalue field in DataImportHandler
> 
>
> Key: SOLR-1612
> URL: https://issues.apache.org/jira/browse/SOLR-1612
> Project: Solr
>  Issue Type: Improvement
>  Components: contrib - DataImportHandler
>Affects Versions: 1.4
>Reporter: Matt Inger
> Fix For: 1.5
>
> Attachments: SOLR-1612.patch
>
>
> I am attaching a patch to the JdbcDataSource class which allows the it to 
> populate a multiValued field from a query which returns a java.sql.Array type 
> for a column value.
> In order to make this work, you need to do the following:
> 1.  In your schema.xml file, set the "multiValued" attribute to the value 
> "true"
>  
> 2.  Have your sql query return a sql type of java.sql.Types.ARRAY.  How to do 
> this is very database dependent, but in oracle, you first create a type:
>  create or replace type FOO_TYPE IS TABLE OF VARCHAR2(255);
>   and then when you select the values, you have to use MULTISET and cast 
> to the created type (assuming here that FOO_PARENT_ID is the condition to
>   limit which foos belong to the current record):
>  CAST(MULTISET(SELECT FOO FROM FOOS WHERE 
> FOO_PARENT=FOO_PARENT_ID) AS FOO_TYPE ) MY_FOOS
> The main advantage this has is it can save you a ton of queries over the 
> current method which will fire off an individual query for each document in 
> order to retrieve the values for each multiValued field.  This method 
> executes much faster, and saves alot of database resources.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-741) Add support for rounding dates in DateField

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796124#action_12796124
 ] 

Noble Paul commented on SOLR-741:
-

With the introduction of Trie fields is it not irrelevant now? can we close it

> Add support for rounding dates in DateField
> ---
>
> Key: SOLR-741
> URL: https://issues.apache.org/jira/browse/SOLR-741
> Project: Solr
>  Issue Type: Improvement
>  Components: search
>Affects Versions: 1.4
>Reporter: Shalin Shekhar Mangar
>Priority: Minor
> Fix For: 1.5
>
>
> As discussed at http://www.nabble.com/Rounding-date-fields-td19203108.html
> Since rounding dates to a coarse value is an often recommended solution to 
> decrease number of unique terms, we should add support for doing this in 
> DateField itself. A number of syntax were proposed, some of them were:
> #  sortMissingLast="true"omitNorms="true" roundTo="-1MINUTE" /> (Shalin)
> #  omitNorms="true" round="DOWN_MINUTE" /> (Otis)
> Hoss proposed more general enhancements related to arbitary pre-processing of 
> values prior to indexing/storing using pre-processing analyzers.
> This issue aims to build a consensus on the solution to pursue and to 
> implement that solution inside Solr.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (SOLR-1212) TestNG Test Case

[Shalin Shekhar Mangar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796123#action_12796123
 ] 

Shalin Shekhar Mangar commented on SOLR-1212:
-

bq. Keeping this out of the codebase would result in the patch being out of 
sync with the tree. If there were no licensing restrictions - what is the harm 
in having this in the tree.

You wrote this because you needed it at work and I appreciate that you thought 
about contributing it to Solr. But from Solr's perspective it is not needed and 
therefore I don't see why we should ship it at all. It is a class that is not 
used by Solr but would need to be maintained by us if we ship it.

> TestNG Test Case 
> -
>
> Key: SOLR-1212
> URL: https://issues.apache.org/jira/browse/SOLR-1212
> Project: Solr
>  Issue Type: New Feature
>  Components: clients - java
>Affects Versions: 1.4
> Environment: Java 6
>Reporter: Kay Kay
> Fix For: 1.5
>
> Attachments: SOLR-1212.patch, testng-5.9-jdk15.jar
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> TestNG equivalent of AbstractSolrTestCase , without using JUnit altogether . 
> New Class created: AbstractSolrNGTest 
> LICENSE.txt , NOTICE.txt modified as appropriate. ( TestNG under Apache 
> License 2.0 ) 
> TestNG 5.9-jdk15 added to lib. 
> Justification:  In some workplaces - people are moving towards TestNG and 
> take out JUnit altogether from the classpath. Hence useful in those cases.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-1644) Provide a clean way to keep flags and helper objects in ResponseBuilder

[Noble Paul (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul updated SOLR-1644:
-

Attachment: SOLR-1644.patch

no changes to any component. The Store field is created. and if any component 
wishes to use it , it can do so. 

> Provide a clean way to keep flags and helper objects in ResponseBuilder
> ---
>
> Key: SOLR-1644
> URL: https://issues.apache.org/jira/browse/SOLR-1644
> Project: Solr
>  Issue Type: Improvement
>  Components: search
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
> Fix For: 1.5
>
> Attachments: SOLR-1644.patch, SOLR-1644.patch, SOLR-1644.patch
>
>
> Many components such as StatsComponent, FacetComponent etc keep flags and 
> helper objects in ResponseBuilder. Having to modify the ResponseBuilder for 
> such things is a very kludgy solution.
> Let us provide a clean way for components to keep arbitrary objects for the 
> duration of a (distributed) search request.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.