Re: change index/store at indexing time

[Lee Carroll]

Vazquez,
Sorry I don't have an answer but I'd love to know what you need this for :-)

I think the logic is going to have to bleed into your search app. In
short copy field and your app knows which to search in.

lee c



On 30 April 2012 20:41, Erick Erickson  wrote:
> OK, I took another look at what you were trying to
> accomplish and, I find the use-case kind of hard to
> figure out, but that's my problem .
>
> But it is true that there's really no good way to _change_ the
> way the field is analyzed in Solr. Of course since Solr is
> built on Lucene, you could to a lot of work and bypass the
> analysis chain, but that seems like more work than this is
> really worth. So using multiple fields seems like the way
> to go.
>
> This is pretty unusual. You have a field that you can
>> search and display the first value
>> search but not display values 2-99
>> display but not search values 100-*
>
> nobody's thought about how to specify this in a schema file,
> and it appears to be an unusual enough use case that
> nobody has proposed anything like that that I know of.
>
> Best
> Erick
>
> On Mon, Apr 30, 2012 at 1:11 PM, Vazquez, Maria (STM)
>  wrote:
>> Thanks Erick.
>> I'm not concerned about the logic, all I want to achieve is sometimes
>> storing/indexing a multi-valued field and sometimes not (same field with
>> same name) based on some logic. In a transformer I cannot change the
>> schema dynamically to do that, not that I know of at least.
>> So if I define that field in the schema.xml I guess I'm stuck with the
>> store/indexed true/false and cannot change that at indexing time.
>> In Lucene you can just do what I had in my original email and the field
>> will be sometimes stored/indexed and sometimes not without a problem.
>>
>> Does that make sense?
>> Thanks,
>> Maria
>>
>>
>>
>> Maria Vazquez  |  Manager Software Engineering  |  Dex One
>> Phone: 310.586.4157
>>
>> www.DexOne.com   |  www.DexKnows.com
>> 
>>  
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On 4/30/12 6:08 AM, "Erick Erickson"  wrote:
>>
>>>Your idea of using a Transformer will work just fine, you have a lot more
>>>flexibility in a custom Transformer, see:
>>>http://wiki.apache.org/solr/DIHCustomTransformer
>>>
>>>You could also write a custom update handler that examined the
>>>document on the server side and implemented your logic, or even
>>>just add an element into an "update chain".
>>>
>>>None of these is very hard, but they can be a bit intimidating
>>>when stating from scratch.
>>>
>>>Best
>>>Erick
>>>
>>>On Sun, Apr 29, 2012 at 12:53 PM, Vazquez, Maria (STM)
>>> wrote:
 Thanks for your response.
 That's what I need, changes at indexing time. Dynamic fields are not
what I need because the field name is the same, I just need to change if
they indexed/stored based on some logic.
 It's so easily achieved with the Lucene API, I was sure there was a way
to do the same in Solr.


 On Apr 28, 2012, at 22:34, "Jeevanandam"  wrote:

> Maria,
>
> thanks for detailed explanation.
> as per schema.xml; stored or indexed should be defined at design-time.
>Per my understanding defining at runtime is not feasible.
> BTW, you can have multiValued="true" attribute for dynamic fields too.
>
> - Jeevanandam
>
> On 29-04-2012 2:06 am, Vazquez, Maria (STM) wrote:
>> Thanks Jeevanandam.
>> That still doesn't have the same behavior as Lucene since multiple
>> fields with different names have to be created.
>> What I want is this exactly (multi-value field)
>>
>> document.add(new Field("geoids", geoId, Field.Store.YES,
>> Field.Index.NOT_ANALYZED_NO_NORMS));
>>
>> document.add(new Field("geoids", geoId, Field.Store.NO,
>> Field.Index.NOT_ANALYZED_NO_NORMS));
>>
>> In Lucene I can save geoids first as stored and in the next line as
>> not stored and it will do exactly that. I want to duplicate this
>> behavior in Solr but I can't do it having only one field in the schema
>> called geoids that I an manipulate at inde time whether to store or
>> not depending on a condition.
>>
>> Thanks again for the help, hope this explanation makes it more clear
>> in what I'm trying to do.
>>
>> Maria
>>
>> On Apr 28, 2012, at 11:49 AM, "Jeevanandam"
>> mailto:je...@myjeeva.com>> wrote:
>>
>> Maria,
>>
>> For your need please define unique pattern using dynamic field in
>>schema.xml
>>
>> Please have a look
>>http://wiki.apache.org/solr/SchemaXml#Dynamic_fields
>>
>> Hope that helps!
>>
>> -Jeevanandam
>>
>> Technology keeps you connected!
>>
>> On Apr 28, 2012, at 10:33 PM, "Vazquez, Maria (STM)"
>> mailto:maria.vazq...@dexone.com>> wrote:
>>
>> I can call a script for the logic part but what I want to figure out
>> is how to save the s

Re: change index/store at indexing time

[Erick Erickson]

OK, I took another look at what you were trying to
accomplish and, I find the use-case kind of hard to
figure out, but that's my problem .

But it is true that there's really no good way to _change_ the
way the field is analyzed in Solr. Of course since Solr is
built on Lucene, you could to a lot of work and bypass the
analysis chain, but that seems like more work than this is
really worth. So using multiple fields seems like the way
to go.

This is pretty unusual. You have a field that you can
> search and display the first value
> search but not display values 2-99
> display but not search values 100-*

nobody's thought about how to specify this in a schema file,
and it appears to be an unusual enough use case that
nobody has proposed anything like that that I know of.

Best
Erick

On Mon, Apr 30, 2012 at 1:11 PM, Vazquez, Maria (STM)
 wrote:
> Thanks Erick.
> I'm not concerned about the logic, all I want to achieve is sometimes
> storing/indexing a multi-valued field and sometimes not (same field with
> same name) based on some logic. In a transformer I cannot change the
> schema dynamically to do that, not that I know of at least.
> So if I define that field in the schema.xml I guess I'm stuck with the
> store/indexed true/false and cannot change that at indexing time.
> In Lucene you can just do what I had in my original email and the field
> will be sometimes stored/indexed and sometimes not without a problem.
>
> Does that make sense?
> Thanks,
> Maria
>
>
>
> Maria Vazquez  |  Manager Software Engineering  |  Dex One
> Phone: 310.586.4157
>
> www.DexOne.com   |  www.DexKnows.com
> 
>  
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On 4/30/12 6:08 AM, "Erick Erickson"  wrote:
>
>>Your idea of using a Transformer will work just fine, you have a lot more
>>flexibility in a custom Transformer, see:
>>http://wiki.apache.org/solr/DIHCustomTransformer
>>
>>You could also write a custom update handler that examined the
>>document on the server side and implemented your logic, or even
>>just add an element into an "update chain".
>>
>>None of these is very hard, but they can be a bit intimidating
>>when stating from scratch.
>>
>>Best
>>Erick
>>
>>On Sun, Apr 29, 2012 at 12:53 PM, Vazquez, Maria (STM)
>> wrote:
>>> Thanks for your response.
>>> That's what I need, changes at indexing time. Dynamic fields are not
>>>what I need because the field name is the same, I just need to change if
>>>they indexed/stored based on some logic.
>>> It's so easily achieved with the Lucene API, I was sure there was a way
>>>to do the same in Solr.
>>>
>>>
>>> On Apr 28, 2012, at 22:34, "Jeevanandam"  wrote:
>>>
 Maria,

 thanks for detailed explanation.
 as per schema.xml; stored or indexed should be defined at design-time.
Per my understanding defining at runtime is not feasible.
 BTW, you can have multiValued="true" attribute for dynamic fields too.

 - Jeevanandam

 On 29-04-2012 2:06 am, Vazquez, Maria (STM) wrote:
> Thanks Jeevanandam.
> That still doesn't have the same behavior as Lucene since multiple
> fields with different names have to be created.
> What I want is this exactly (multi-value field)
>
> document.add(new Field("geoids", geoId, Field.Store.YES,
> Field.Index.NOT_ANALYZED_NO_NORMS));
>
> document.add(new Field("geoids", geoId, Field.Store.NO,
> Field.Index.NOT_ANALYZED_NO_NORMS));
>
> In Lucene I can save geoids first as stored and in the next line as
> not stored and it will do exactly that. I want to duplicate this
> behavior in Solr but I can't do it having only one field in the schema
> called geoids that I an manipulate at inde time whether to store or
> not depending on a condition.
>
> Thanks again for the help, hope this explanation makes it more clear
> in what I'm trying to do.
>
> Maria
>
> On Apr 28, 2012, at 11:49 AM, "Jeevanandam"
> mailto:je...@myjeeva.com>> wrote:
>
> Maria,
>
> For your need please define unique pattern using dynamic field in
>schema.xml
>
> Please have a look
>http://wiki.apache.org/solr/SchemaXml#Dynamic_fields
>
> Hope that helps!
>
> -Jeevanandam
>
> Technology keeps you connected!
>
> On Apr 28, 2012, at 10:33 PM, "Vazquez, Maria (STM)"
> mailto:maria.vazq...@dexone.com>> wrote:
>
> I can call a script for the logic part but what I want to figure out
> is how to save the same field sometimes as stored and indexed,
> sometimes as stored not indexed, etc. From a transformer or a script I
> didn't see anything where I can modify that at indexing time.
> Thanks a lot,
> Maria
>
>
> On Apr 27, 2012, at 18:38, "Bill Bell"
> mailto:billnb...@gmail.com>> wrote:
>
> Yes you can. Just use a script that is called for each row.
>
> Bill Bell
> Sent from m

Re: change index/store at indexing time

[Vazquez, Maria (STM)]

Thanks Erick.
I'm not concerned about the logic, all I want to achieve is sometimes
storing/indexing a multi-valued field and sometimes not (same field with
same name) based on some logic. In a transformer I cannot change the
schema dynamically to do that, not that I know of at least.
So if I define that field in the schema.xml I guess I'm stuck with the
store/indexed true/false and cannot change that at indexing time.
In Lucene you can just do what I had in my original email and the field
will be sometimes stored/indexed and sometimes not without a problem.

Does that make sense?
Thanks,
Maria



Maria Vazquez  |  Manager Software Engineering  |  Dex One
Phone: 310.586.4157

www.DexOne.com   |  www.DexKnows.com

 















On 4/30/12 6:08 AM, "Erick Erickson"  wrote:

>Your idea of using a Transformer will work just fine, you have a lot more
>flexibility in a custom Transformer, see:
>http://wiki.apache.org/solr/DIHCustomTransformer
>
>You could also write a custom update handler that examined the
>document on the server side and implemented your logic, or even
>just add an element into an "update chain".
>
>None of these is very hard, but they can be a bit intimidating
>when stating from scratch.
>
>Best
>Erick
>
>On Sun, Apr 29, 2012 at 12:53 PM, Vazquez, Maria (STM)
> wrote:
>> Thanks for your response.
>> That's what I need, changes at indexing time. Dynamic fields are not
>>what I need because the field name is the same, I just need to change if
>>they indexed/stored based on some logic.
>> It's so easily achieved with the Lucene API, I was sure there was a way
>>to do the same in Solr.
>>
>>
>> On Apr 28, 2012, at 22:34, "Jeevanandam"  wrote:
>>
>>> Maria,
>>>
>>> thanks for detailed explanation.
>>> as per schema.xml; stored or indexed should be defined at design-time.
>>>Per my understanding defining at runtime is not feasible.
>>> BTW, you can have multiValued="true" attribute for dynamic fields too.
>>>
>>> - Jeevanandam
>>>
>>> On 29-04-2012 2:06 am, Vazquez, Maria (STM) wrote:
 Thanks Jeevanandam.
 That still doesn't have the same behavior as Lucene since multiple
 fields with different names have to be created.
 What I want is this exactly (multi-value field)

 document.add(new Field("geoids", geoId, Field.Store.YES,
 Field.Index.NOT_ANALYZED_NO_NORMS));

 document.add(new Field("geoids", geoId, Field.Store.NO,
 Field.Index.NOT_ANALYZED_NO_NORMS));

 In Lucene I can save geoids first as stored and in the next line as
 not stored and it will do exactly that. I want to duplicate this
 behavior in Solr but I can't do it having only one field in the schema
 called geoids that I an manipulate at inde time whether to store or
 not depending on a condition.

 Thanks again for the help, hope this explanation makes it more clear
 in what I'm trying to do.

 Maria

 On Apr 28, 2012, at 11:49 AM, "Jeevanandam"
 mailto:je...@myjeeva.com>> wrote:

 Maria,

 For your need please define unique pattern using dynamic field in
schema.xml

 Please have a look
http://wiki.apache.org/solr/SchemaXml#Dynamic_fields

 Hope that helps!

 -Jeevanandam

 Technology keeps you connected!

 On Apr 28, 2012, at 10:33 PM, "Vazquez, Maria (STM)"
 mailto:maria.vazq...@dexone.com>> wrote:

 I can call a script for the logic part but what I want to figure out
 is how to save the same field sometimes as stored and indexed,
 sometimes as stored not indexed, etc. From a transformer or a script I
 didn't see anything where I can modify that at indexing time.
 Thanks a lot,
 Maria


 On Apr 27, 2012, at 18:38, "Bill Bell"
 mailto:billnb...@gmail.com>> wrote:

 Yes you can. Just use a script that is called for each row.

 Bill Bell
 Sent from mobile


 On Apr 27, 2012, at 6:38 PM, "Vazquez, Maria (STM)"
 mailto:maria.vazq...@dexone.com>> wrote:

 Hi,
 I'm migrating a project from Lucene 2.9 to Solr 3.4.
 There is a special case in the code that indexes the same field in
 two different ways, which is completely legal in Lucene directly but I
 don't know how to duplicate this same behavior in Solr:

 if (isFirstGeo) {
 document.add(new Field("geoids", geoId, Field.Store.YES,
 Field.Index.NOT_ANALYZED_NO_NORMS));
 isFirstGeo = false;
 } else {
 if (countProducts < 100)
  document.add(new Field("geoids", geoId, Field.Store.NO,
 Field.Index.NOT_ANALYZED_NO_NORMS));
 else
  document.add(new Field("geoids", geoId, Field.Store.YES,
 Field.Index.NO));
 }

 Is there any way to do this in Solr in a Tranformer? I'm using the
 DIH to index and I can't see a way to do this other than having three
 fields in the schema like geoids_s

Re: change index/store at indexing time

[Erick Erickson]

Your idea of using a Transformer will work just fine, you have a lot more
flexibility in a custom Transformer, see:
http://wiki.apache.org/solr/DIHCustomTransformer

You could also write a custom update handler that examined the
document on the server side and implemented your logic, or even
just add an element into an "update chain".

None of these is very hard, but they can be a bit intimidating
when stating from scratch.

Best
Erick

On Sun, Apr 29, 2012 at 12:53 PM, Vazquez, Maria (STM)
 wrote:
> Thanks for your response.
> That's what I need, changes at indexing time. Dynamic fields are not what I 
> need because the field name is the same, I just need to change if they 
> indexed/stored based on some logic.
> It's so easily achieved with the Lucene API, I was sure there was a way to do 
> the same in Solr.
>
>
> On Apr 28, 2012, at 22:34, "Jeevanandam"  wrote:
>
>> Maria,
>>
>> thanks for detailed explanation.
>> as per schema.xml; stored or indexed should be defined at design-time. Per 
>> my understanding defining at runtime is not feasible.
>> BTW, you can have multiValued="true" attribute for dynamic fields too.
>>
>> - Jeevanandam
>>
>> On 29-04-2012 2:06 am, Vazquez, Maria (STM) wrote:
>>> Thanks Jeevanandam.
>>> That still doesn't have the same behavior as Lucene since multiple
>>> fields with different names have to be created.
>>> What I want is this exactly (multi-value field)
>>>
>>> document.add(new Field("geoids", geoId, Field.Store.YES,
>>> Field.Index.NOT_ANALYZED_NO_NORMS));
>>>
>>> document.add(new Field("geoids", geoId, Field.Store.NO,
>>> Field.Index.NOT_ANALYZED_NO_NORMS));
>>>
>>> In Lucene I can save geoids first as stored and in the next line as
>>> not stored and it will do exactly that. I want to duplicate this
>>> behavior in Solr but I can't do it having only one field in the schema
>>> called geoids that I an manipulate at inde time whether to store or
>>> not depending on a condition.
>>>
>>> Thanks again for the help, hope this explanation makes it more clear
>>> in what I'm trying to do.
>>>
>>> Maria
>>>
>>> On Apr 28, 2012, at 11:49 AM, "Jeevanandam"
>>> mailto:je...@myjeeva.com>> wrote:
>>>
>>> Maria,
>>>
>>> For your need please define unique pattern using dynamic field in schema.xml
>>>
>>> Please have a look http://wiki.apache.org/solr/SchemaXml#Dynamic_fields
>>>
>>> Hope that helps!
>>>
>>> -Jeevanandam
>>>
>>> Technology keeps you connected!
>>>
>>> On Apr 28, 2012, at 10:33 PM, "Vazquez, Maria (STM)"
>>> mailto:maria.vazq...@dexone.com>> wrote:
>>>
>>> I can call a script for the logic part but what I want to figure out
>>> is how to save the same field sometimes as stored and indexed,
>>> sometimes as stored not indexed, etc. From a transformer or a script I
>>> didn't see anything where I can modify that at indexing time.
>>> Thanks a lot,
>>> Maria
>>>
>>>
>>> On Apr 27, 2012, at 18:38, "Bill Bell"
>>> mailto:billnb...@gmail.com>> wrote:
>>>
>>> Yes you can. Just use a script that is called for each row.
>>>
>>> Bill Bell
>>> Sent from mobile
>>>
>>>
>>> On Apr 27, 2012, at 6:38 PM, "Vazquez, Maria (STM)"
>>> mailto:maria.vazq...@dexone.com>> wrote:
>>>
>>> Hi,
>>> I'm migrating a project from Lucene 2.9 to Solr 3.4.
>>> There is a special case in the code that indexes the same field in
>>> two different ways, which is completely legal in Lucene directly but I
>>> don't know how to duplicate this same behavior in Solr:
>>>
>>> if (isFirstGeo) {
>>> document.add(new Field("geoids", geoId, Field.Store.YES,
>>> Field.Index.NOT_ANALYZED_NO_NORMS));
>>> isFirstGeo = false;
>>> } else {
>>> if (countProducts < 100)
>>>      document.add(new Field("geoids", geoId, Field.Store.NO,
>>> Field.Index.NOT_ANALYZED_NO_NORMS));
>>> else
>>>      document.add(new Field("geoids", geoId, Field.Store.YES,
>>> Field.Index.NO));
>>> }
>>>
>>> Is there any way to do this in Solr in a Tranformer? I'm using the
>>> DIH to index and I can't see a way to do this other than having three
>>> fields in the schema like geoids_store_index, geoids_nostore_index,
>>> and geoids_store_noindex.
>>>
>>> Thanks a lot in advance.
>>> Maria
>>

Re: change index/store at indexing time

[Vazquez, Maria (STM)]

Thanks for your response.
That's what I need, changes at indexing time. Dynamic fields are not what I 
need because the field name is the same, I just need to change if they 
indexed/stored based on some logic.
It's so easily achieved with the Lucene API, I was sure there was a way to do 
the same in Solr.


On Apr 28, 2012, at 22:34, "Jeevanandam"  wrote:

> Maria,
> 
> thanks for detailed explanation.
> as per schema.xml; stored or indexed should be defined at design-time. Per my 
> understanding defining at runtime is not feasible.
> BTW, you can have multiValued="true" attribute for dynamic fields too.
> 
> - Jeevanandam
> 
> On 29-04-2012 2:06 am, Vazquez, Maria (STM) wrote:
>> Thanks Jeevanandam.
>> That still doesn't have the same behavior as Lucene since multiple
>> fields with different names have to be created.
>> What I want is this exactly (multi-value field)
>> 
>> document.add(new Field("geoids", geoId, Field.Store.YES,
>> Field.Index.NOT_ANALYZED_NO_NORMS));
>> 
>> document.add(new Field("geoids", geoId, Field.Store.NO,
>> Field.Index.NOT_ANALYZED_NO_NORMS));
>> 
>> In Lucene I can save geoids first as stored and in the next line as
>> not stored and it will do exactly that. I want to duplicate this
>> behavior in Solr but I can't do it having only one field in the schema
>> called geoids that I an manipulate at inde time whether to store or
>> not depending on a condition.
>> 
>> Thanks again for the help, hope this explanation makes it more clear
>> in what I'm trying to do.
>> 
>> Maria
>> 
>> On Apr 28, 2012, at 11:49 AM, "Jeevanandam"
>> mailto:je...@myjeeva.com>> wrote:
>> 
>> Maria,
>> 
>> For your need please define unique pattern using dynamic field in schema.xml
>> 
>> Please have a look http://wiki.apache.org/solr/SchemaXml#Dynamic_fields
>> 
>> Hope that helps!
>> 
>> -Jeevanandam
>> 
>> Technology keeps you connected!
>> 
>> On Apr 28, 2012, at 10:33 PM, "Vazquez, Maria (STM)"
>> mailto:maria.vazq...@dexone.com>> wrote:
>> 
>> I can call a script for the logic part but what I want to figure out
>> is how to save the same field sometimes as stored and indexed,
>> sometimes as stored not indexed, etc. From a transformer or a script I
>> didn't see anything where I can modify that at indexing time.
>> Thanks a lot,
>> Maria
>> 
>> 
>> On Apr 27, 2012, at 18:38, "Bill Bell"
>> mailto:billnb...@gmail.com>> wrote:
>> 
>> Yes you can. Just use a script that is called for each row.
>> 
>> Bill Bell
>> Sent from mobile
>> 
>> 
>> On Apr 27, 2012, at 6:38 PM, "Vazquez, Maria (STM)"
>> mailto:maria.vazq...@dexone.com>> wrote:
>> 
>> Hi,
>> I'm migrating a project from Lucene 2.9 to Solr 3.4.
>> There is a special case in the code that indexes the same field in
>> two different ways, which is completely legal in Lucene directly but I
>> don't know how to duplicate this same behavior in Solr:
>> 
>> if (isFirstGeo) {
>> document.add(new Field("geoids", geoId, Field.Store.YES,
>> Field.Index.NOT_ANALYZED_NO_NORMS));
>> isFirstGeo = false;
>> } else {
>> if (countProducts < 100)
>>  document.add(new Field("geoids", geoId, Field.Store.NO,
>> Field.Index.NOT_ANALYZED_NO_NORMS));
>> else
>>  document.add(new Field("geoids", geoId, Field.Store.YES,
>> Field.Index.NO));
>> }
>> 
>> Is there any way to do this in Solr in a Tranformer? I'm using the
>> DIH to index and I can't see a way to do this other than having three
>> fields in the schema like geoids_store_index, geoids_nostore_index,
>> and geoids_store_noindex.
>> 
>> Thanks a lot in advance.
>> Maria
> 

Re: change index/store at indexing time

[Jeevanandam Madanagopal]

Maria -

thanks for detailed explanation. 
as per schema.xml; stored or indexed should be defined at schema design itself. 
 as per my understanding defining at runtime is not feasible. 
BTW, you can have multiValued="true" attribute for dynamic fields too.

- Jeevanandam

On Apr 29, 2012, at 1:06 AM, Vazquez, Maria (STM) wrote:

> Thanks Jeevanandam.
> That still doesn't have the same behavior as Lucene since multiple fields 
> with different names have to be created.
> What I want is this exactly (multi-value field)
> 
> document.add(new Field("geoids", geoId, Field.Store.YES, 
> Field.Index.NOT_ANALYZED_NO_NORMS));
> 
> document.add(new Field("geoids", geoId, Field.Store.NO, 
> Field.Index.NOT_ANALYZED_NO_NORMS));
> 
> In Lucene I can save geoids first as stored and in the next line as not 
> stored and it will do exactly that. I want to duplicate this behavior in Solr 
> but I can't do it having only one field in the schema called geoids that I an 
> manipulate at inde time whether to store or not depending on a condition.
> 
> Thanks again for the help, hope this explanation makes it more clear in what 
> I'm trying to do.
> 
> Maria
> 
> On Apr 28, 2012, at 11:49 AM, "Jeevanandam" 
> mailto:je...@myjeeva.com>> wrote:
> 
> Maria,
> 
> For your need please define unique pattern using dynamic field in schema.xml
> 
> Please have a look http://wiki.apache.org/solr/SchemaXml#Dynamic_fields
> 
> Hope that helps!
> 
> -Jeevanandam
> 
> Technology keeps you connected!
> 
> On Apr 28, 2012, at 10:33 PM, "Vazquez, Maria (STM)" 
> mailto:maria.vazq...@dexone.com>> wrote:
> 
> I can call a script for the logic part but what I want to figure out is how 
> to save the same field sometimes as stored and indexed, sometimes as stored 
> not indexed, etc. From a transformer or a script I didn't see anything where 
> I can modify that at indexing time.
> Thanks a lot,
> Maria
> 
> 
> On Apr 27, 2012, at 18:38, "Bill Bell" 
> mailto:billnb...@gmail.com>> wrote:
> 
> Yes you can. Just use a script that is called for each row.
> 
> Bill Bell
> Sent from mobile
> 
> 
> On Apr 27, 2012, at 6:38 PM, "Vazquez, Maria (STM)" 
> mailto:maria.vazq...@dexone.com>> wrote:
> 
> Hi,
> I'm migrating a project from Lucene 2.9 to Solr 3.4.
> There is a special case in the code that indexes the same field in two 
> different ways, which is completely legal in Lucene directly but I don't know 
> how to duplicate this same behavior in Solr:
> 
> if (isFirstGeo) {
> document.add(new Field("geoids", geoId, Field.Store.YES, 
> Field.Index.NOT_ANALYZED_NO_NORMS));
> isFirstGeo = false;
> } else {
> if (countProducts < 100)
>document.add(new Field("geoids", geoId, Field.Store.NO, 
> Field.Index.NOT_ANALYZED_NO_NORMS));
> else
>document.add(new Field("geoids", geoId, Field.Store.YES, Field.Index.NO));
> }
> 
> Is there any way to do this in Solr in a Tranformer? I'm using the DIH to 
> index and I can't see a way to do this other than having three fields in the 
> schema like geoids_store_index, geoids_nostore_index, and 
> geoids_store_noindex.
> 
> Thanks a lot in advance.
> Maria
> 
> 
> 

Re: change index/store at indexing time

[Jeevanandam]

Maria,

thanks for detailed explanation.
as per schema.xml; stored or indexed should be defined at design-time. 
Per my understanding defining at runtime is not feasible.

BTW, you can have multiValued="true" attribute for dynamic fields too.

- Jeevanandam

On 29-04-2012 2:06 am, Vazquez, Maria (STM) wrote:

Thanks Jeevanandam.
That still doesn't have the same behavior as Lucene since multiple
fields with different names have to be created.
What I want is this exactly (multi-value field)

document.add(new Field("geoids", geoId, Field.Store.YES,
Field.Index.NOT_ANALYZED_NO_NORMS));

document.add(new Field("geoids", geoId, Field.Store.NO,
Field.Index.NOT_ANALYZED_NO_NORMS));

In Lucene I can save geoids first as stored and in the next line as
not stored and it will do exactly that. I want to duplicate this
behavior in Solr but I can't do it having only one field in the 
schema

called geoids that I an manipulate at inde time whether to store or
not depending on a condition.

Thanks again for the help, hope this explanation makes it more clear
in what I'm trying to do.

Maria

On Apr 28, 2012, at 11:49 AM, "Jeevanandam"
mailto:je...@myjeeva.com>> wrote:

Maria,

For your need please define unique pattern using dynamic field in 
schema.xml


Please have a look 
http://wiki.apache.org/solr/SchemaXml#Dynamic_fields


Hope that helps!

-Jeevanandam

Technology keeps you connected!

On Apr 28, 2012, at 10:33 PM, "Vazquez, Maria (STM)"
mailto:maria.vazq...@dexone.com>> wrote:

I can call a script for the logic part but what I want to figure out
is how to save the same field sometimes as stored and indexed,
sometimes as stored not indexed, etc. From a transformer or a script 
I

didn't see anything where I can modify that at indexing time.
Thanks a lot,
Maria


On Apr 27, 2012, at 18:38, "Bill Bell"
mailto:billnb...@gmail.com>> wrote:

Yes you can. Just use a script that is called for each row.

Bill Bell
Sent from mobile


On Apr 27, 2012, at 6:38 PM, "Vazquez, Maria (STM)"
mailto:maria.vazq...@dexone.com>> wrote:

Hi,
I'm migrating a project from Lucene 2.9 to Solr 3.4.
There is a special case in the code that indexes the same field in
two different ways, which is completely legal in Lucene directly but 
I

don't know how to duplicate this same behavior in Solr:

if (isFirstGeo) {
 document.add(new Field("geoids", geoId, Field.Store.YES,
Field.Index.NOT_ANALYZED_NO_NORMS));
 isFirstGeo = false;
} else {
 if (countProducts < 100)
  document.add(new Field("geoids", geoId, Field.Store.NO,
Field.Index.NOT_ANALYZED_NO_NORMS));
 else
  document.add(new Field("geoids", geoId, Field.Store.YES,
Field.Index.NO));
}

Is there any way to do this in Solr in a Tranformer? I'm using the
DIH to index and I can't see a way to do this other than having three
fields in the schema like geoids_store_index, geoids_nostore_index,
and geoids_store_noindex.

Thanks a lot in advance.
Maria

Re: change index/store at indexing time

[Vazquez, Maria (STM)]

Thanks Jeevanandam.
That still doesn't have the same behavior as Lucene since multiple fields with 
different names have to be created.
What I want is this exactly (multi-value field)

document.add(new Field("geoids", geoId, Field.Store.YES, 
Field.Index.NOT_ANALYZED_NO_NORMS));

document.add(new Field("geoids", geoId, Field.Store.NO, 
Field.Index.NOT_ANALYZED_NO_NORMS));

In Lucene I can save geoids first as stored and in the next line as not stored 
and it will do exactly that. I want to duplicate this behavior in Solr but I 
can't do it having only one field in the schema called geoids that I an 
manipulate at inde time whether to store or not depending on a condition.

Thanks again for the help, hope this explanation makes it more clear in what 
I'm trying to do.

Maria

On Apr 28, 2012, at 11:49 AM, "Jeevanandam" 
mailto:je...@myjeeva.com>> wrote:

Maria,

For your need please define unique pattern using dynamic field in schema.xml

Please have a look http://wiki.apache.org/solr/SchemaXml#Dynamic_fields

Hope that helps!

-Jeevanandam

Technology keeps you connected!

On Apr 28, 2012, at 10:33 PM, "Vazquez, Maria (STM)" 
mailto:maria.vazq...@dexone.com>> wrote:

I can call a script for the logic part but what I want to figure out is how to 
save the same field sometimes as stored and indexed, sometimes as stored not 
indexed, etc. From a transformer or a script I didn't see anything where I can 
modify that at indexing time.
Thanks a lot,
Maria


On Apr 27, 2012, at 18:38, "Bill Bell" 
mailto:billnb...@gmail.com>> wrote:

Yes you can. Just use a script that is called for each row.

Bill Bell
Sent from mobile


On Apr 27, 2012, at 6:38 PM, "Vazquez, Maria (STM)" 
mailto:maria.vazq...@dexone.com>> wrote:

Hi,
I'm migrating a project from Lucene 2.9 to Solr 3.4.
There is a special case in the code that indexes the same field in two 
different ways, which is completely legal in Lucene directly but I don't know 
how to duplicate this same behavior in Solr:

if (isFirstGeo) {
 document.add(new Field("geoids", geoId, Field.Store.YES, 
Field.Index.NOT_ANALYZED_NO_NORMS));
 isFirstGeo = false;
} else {
 if (countProducts < 100)
  document.add(new Field("geoids", geoId, Field.Store.NO, 
Field.Index.NOT_ANALYZED_NO_NORMS));
 else
  document.add(new Field("geoids", geoId, Field.Store.YES, Field.Index.NO));
}

Is there any way to do this in Solr in a Tranformer? I'm using the DIH to index 
and I can't see a way to do this other than having three fields in the schema 
like geoids_store_index, geoids_nostore_index, and geoids_store_noindex.

Thanks a lot in advance.
Maria

Re: change index/store at indexing time

[Jeevanandam]

Maria,

For your need please define unique pattern using dynamic field in schema.xml

Please have a look http://wiki.apache.org/solr/SchemaXml#Dynamic_fields

Hope that helps!

-Jeevanandam

Technology keeps you connected!

On Apr 28, 2012, at 10:33 PM, "Vazquez, Maria (STM)"  
wrote:

> I can call a script for the logic part but what I want to figure out is how 
> to save the same field sometimes as stored and indexed, sometimes as stored 
> not indexed, etc. From a transformer or a script I didn't see anything where 
> I can modify that at indexing time.
> Thanks a lot,
> Maria
> 
> 
> On Apr 27, 2012, at 18:38, "Bill Bell"  wrote:
> 
>> Yes you can. Just use a script that is called for each row.
>> 
>> Bill Bell
>> Sent from mobile
>> 
>> 
>> On Apr 27, 2012, at 6:38 PM, "Vazquez, Maria (STM)" 
>>  wrote:
>> 
>>> Hi,
>>> I'm migrating a project from Lucene 2.9 to Solr 3.4.
>>> There is a special case in the code that indexes the same field in two 
>>> different ways, which is completely legal in Lucene directly but I don't 
>>> know how to duplicate this same behavior in Solr:
>>> 
>>> if (isFirstGeo) {
>>>   document.add(new Field("geoids", geoId, Field.Store.YES, 
>>> Field.Index.NOT_ANALYZED_NO_NORMS));
>>>   isFirstGeo = false;
>>> } else {
>>>   if (countProducts < 100)
>>>document.add(new Field("geoids", geoId, Field.Store.NO, 
>>> Field.Index.NOT_ANALYZED_NO_NORMS));
>>>   else
>>>document.add(new Field("geoids", geoId, Field.Store.YES, 
>>> Field.Index.NO));
>>> }
>>> 
>>> Is there any way to do this in Solr in a Tranformer? I'm using the DIH to 
>>> index and I can't see a way to do this other than having three fields in 
>>> the schema like geoids_store_index, geoids_nostore_index, and 
>>> geoids_store_noindex.
>>> 
>>> Thanks a lot in advance.
>>> Maria
>>> 
>>> 
>>> 

Re: change index/store at indexing time

[Vazquez, Maria (STM)]

I can call a script for the logic part but what I want to figure out is how to 
save the same field sometimes as stored and indexed, sometimes as stored not 
indexed, etc. From a transformer or a script I didn't see anything where I can 
modify that at indexing time.
Thanks a lot,
Maria


On Apr 27, 2012, at 18:38, "Bill Bell"  wrote:

> Yes you can. Just use a script that is called for each row.
> 
> Bill Bell
> Sent from mobile
> 
> 
> On Apr 27, 2012, at 6:38 PM, "Vazquez, Maria (STM)" 
>  wrote:
> 
>> Hi,
>> I'm migrating a project from Lucene 2.9 to Solr 3.4.
>> There is a special case in the code that indexes the same field in two 
>> different ways, which is completely legal in Lucene directly but I don't 
>> know how to duplicate this same behavior in Solr:
>> 
>> if (isFirstGeo) {
>>document.add(new Field("geoids", geoId, Field.Store.YES, 
>> Field.Index.NOT_ANALYZED_NO_NORMS));
>>isFirstGeo = false;
>> } else {
>>if (countProducts < 100)
>> document.add(new Field("geoids", geoId, Field.Store.NO, 
>> Field.Index.NOT_ANALYZED_NO_NORMS));
>>else
>> document.add(new Field("geoids", geoId, Field.Store.YES, 
>> Field.Index.NO));
>> }
>> 
>> Is there any way to do this in Solr in a Tranformer? I'm using the DIH to 
>> index and I can't see a way to do this other than having three fields in the 
>> schema like geoids_store_index, geoids_nostore_index, and 
>> geoids_store_noindex.
>> 
>> Thanks a lot in advance.
>> Maria
>> 
>> 
>> 

Re: change index/store at indexing time

[Jeevanandam Madanagopal]

As Bill mentioned, you can use Script. 
Please have a look: 
http://wiki.apache.org/solr/DataImportHandler#ScriptTransformer

- Jeevanandam

On Apr 28, 2012, at 7:07 AM, Bill Bell wrote:

> Yes you can. Just use a script that is called for each row.
> 
> Bill Bell
> Sent from mobile
> 
> 
> On Apr 27, 2012, at 6:38 PM, "Vazquez, Maria (STM)" 
>  wrote:
> 
>> Hi,
>> I'm migrating a project from Lucene 2.9 to Solr 3.4.
>> There is a special case in the code that indexes the same field in two 
>> different ways, which is completely legal in Lucene directly but I don't 
>> know how to duplicate this same behavior in Solr:
>> 
>> if (isFirstGeo) {
>>document.add(new Field("geoids", geoId, Field.Store.YES, 
>> Field.Index.NOT_ANALYZED_NO_NORMS));
>>isFirstGeo = false;
>> } else {
>>if (countProducts < 100)
>> document.add(new Field("geoids", geoId, Field.Store.NO, 
>> Field.Index.NOT_ANALYZED_NO_NORMS));
>>else
>> document.add(new Field("geoids", geoId, Field.Store.YES, 
>> Field.Index.NO));
>> }
>> 
>> Is there any way to do this in Solr in a Tranformer? I'm using the DIH to 
>> index and I can't see a way to do this other than having three fields in the 
>> schema like geoids_store_index, geoids_nostore_index, and 
>> geoids_store_noindex.
>> 
>> Thanks a lot in advance.
>> Maria
>> 
>> 
>> 

Re: change index/store at indexing time

[Bill Bell]

Yes you can. Just use a script that is called for each row.

Bill Bell
Sent from mobile


On Apr 27, 2012, at 6:38 PM, "Vazquez, Maria (STM)"  
wrote:

> Hi,
> I'm migrating a project from Lucene 2.9 to Solr 3.4.
> There is a special case in the code that indexes the same field in two 
> different ways, which is completely legal in Lucene directly but I don't know 
> how to duplicate this same behavior in Solr:
> 
> if (isFirstGeo) {
> document.add(new Field("geoids", geoId, Field.Store.YES, 
> Field.Index.NOT_ANALYZED_NO_NORMS));
> isFirstGeo = false;
> } else {
> if (countProducts < 100)
>  document.add(new Field("geoids", geoId, Field.Store.NO, 
> Field.Index.NOT_ANALYZED_NO_NORMS));
> else
>  document.add(new Field("geoids", geoId, Field.Store.YES, 
> Field.Index.NO));
> }
> 
> Is there any way to do this in Solr in a Tranformer? I'm using the DIH to 
> index and I can't see a way to do this other than having three fields in the 
> schema like geoids_store_index, geoids_nostore_index, and 
> geoids_store_noindex.
> 
> Thanks a lot in advance.
> Maria
> 
> 
> 

change index/store at indexing time

[Vazquez, Maria (STM)]

Hi,
I'm migrating a project from Lucene 2.9 to Solr 3.4.
There is a special case in the code that indexes the same field in two 
different ways, which is completely legal in Lucene directly but I don't know 
how to duplicate this same behavior in Solr:


if (isFirstGeo) {

document.add(new Field("geoids", geoId, Field.Store.YES, 
Field.Index.NOT_ANALYZED_NO_NORMS));

isFirstGeo = false;

} else {

if (countProducts < 100)

 document.add(new Field("geoids", geoId, Field.Store.NO, 
Field.Index.NOT_ANALYZED_NO_NORMS));

else

 document.add(new Field("geoids", geoId, Field.Store.YES, 
Field.Index.NO));

}

Is there any way to do this in Solr in a Tranformer? I'm using the DIH to index 
and I can't see a way to do this other than having three fields in the schema 
like geoids_store_index, geoids_nostore_index, and geoids_store_noindex.

Thanks a lot in advance.
Maria

[cid:74B38F9C-DD1D-40C6-A4DE-F47F201A5DF3]