RE: suggestion for new custom atomic update
Shalin and Matthew, Thank you very much. -Original Message- From: Matthew Nigl [mailto:matthew.n...@gmail.com] Sent: Monday, October 27, 2014 7:24 PM To: solr-user@lucene.apache.org Subject: Re: suggestion for new custom atomic update No problem Elran. As Shalin mentioned, you will need to do it like this: processor class=solr.DistributedUpdateProcessorFactory/ processor class=mycode.solr_plugins.FieldManipulationProcessorFactory / processor class=solr.LogUpdateProcessorFactory / processor class=solr.RunUpdateProcessorFactory / On 28 October 2014 03:22, Shalin Shekhar Mangar shalinman...@gmail.com wrote: Hi Elran, You need to explicitly specify the DistributedUpdateProcessorFactory in the chain and then add your custom processor after it. On Mon, Oct 27, 2014 at 9:26 PM, Elran Dvir elr...@checkpoint.com wrote: Thank you very much for your suggestion. I created an update processor factory with my logic. I changed the update processor chain to be: processor class=solr.LogUpdateProcessorFactory / processor class=solr.RunUpdateProcessorFactory / processor class=mycode.solr_plugins.FieldManipulationProcessorFactory / But nothing seems to happen. When I move my class to be the first in the chain, the logic is running (not as I want, of course. It's calculated based on the update value rather than the stored value) . How can I define a custom update processor factory that will run after DistributedUpdateProcessorFactory? Thank you very much. -Original Message- From: Matthew Nigl [mailto:matthew.n...@gmail.com] Sent: Monday, October 27, 2014 12:10 PM To: solr-user@lucene.apache.org Subject: Re: suggestion for new custom atomic update You can get the summed value, 13, if you add a processor after DistributedUpdateProcessorFactory in the URP chain. Then one possibility would be to clone this value to another field, such as field_b, and run other processors on that field. Or for something more customized, you can use the StatelessScriptUpdateProcessorFactory, and retrieve the value of field_a with: var doc = cmd.solrDoc; // org.apache.solr.common.SolrInputDocument var field_a = doc.getFieldValue(field_a); Note if you try to get the value of field_a before DistributedUpdateProcessorFactory, then using your example with an atomic update, the value would be 5 (the value of the increment from the input document). On 27 October 2014 18:03, Elran Dvir elr...@checkpoint.com wrote: I will explain with an example. Let's say field_a is sent in the update with the value of 5. field_a is already stored in the document with the value 8. After the update field_a should have the value 13 (sum). The value of field_b will be based on the value of 13 and not 5. Is there a way in URP to know what is the value which is already stored in field_a? Thank you very much. -Original Message- From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] Sent: Sunday, October 26, 2014 6:07 PM To: solr-user Subject: Re: suggestion for new custom atomic update I am not sure what the problem is. URP catches all operations. So, you can modify the source document to add the calculation when the field_a is either new or updated. Or are you trying to calculate things across multiple documents? In that case, neither mine nor your solution will work, I think. Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 On 26 October 2014 12:00, Elran Dvir elr...@checkpoint.com wrote: Thanks for your response. If the calculation is based on the most recent summed value of field_a and the value of field_a in the update, how can I? Thanks. -Original Message- From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] Sent: Sunday, October 26, 2014 2:11 PM To: solr-user Subject: RE: suggestion for new custom atomic update Can't you do the calculation in custom UpdateRequestProcessor? Regards, Alex On 26/10/2014 4:17 am, Elran Dvir elr...@checkpoint.com wrote: Hi all, Did anyone have a chance to review my idea? Thanks. -Original Message- From: Elran Dvir Sent: Monday, October 20, 2014 12:42 PM To: solr-user Subject: suggestion for new custom atomic update Hi all, This is my use case: I have a stored field, field_a, which is atomic updated (let's say by inc). field_a is stored but not indexed due to the large number of distinct values it can have. I need to index field_b (I need facet and stats on it) which is not in the document but its value is based on a calculation
RE: suggestion for new custom atomic update
I will explain with an example. Let's say field_a is sent in the update with the value of 5. field_a is already stored in the document with the value 8. After the update field_a should have the value 13 (sum). The value of field_b will be based on the value of 13 and not 5. Is there a way in URP to know what is the value which is already stored in field_a? Thank you very much. -Original Message- From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] Sent: Sunday, October 26, 2014 6:07 PM To: solr-user Subject: Re: suggestion for new custom atomic update I am not sure what the problem is. URP catches all operations. So, you can modify the source document to add the calculation when the field_a is either new or updated. Or are you trying to calculate things across multiple documents? In that case, neither mine nor your solution will work, I think. Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 On 26 October 2014 12:00, Elran Dvir elr...@checkpoint.com wrote: Thanks for your response. If the calculation is based on the most recent summed value of field_a and the value of field_a in the update, how can I? Thanks. -Original Message- From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] Sent: Sunday, October 26, 2014 2:11 PM To: solr-user Subject: RE: suggestion for new custom atomic update Can't you do the calculation in custom UpdateRequestProcessor? Regards, Alex On 26/10/2014 4:17 am, Elran Dvir elr...@checkpoint.com wrote: Hi all, Did anyone have a chance to review my idea? Thanks. -Original Message- From: Elran Dvir Sent: Monday, October 20, 2014 12:42 PM To: solr-user Subject: suggestion for new custom atomic update Hi all, This is my use case: I have a stored field, field_a, which is atomic updated (let's say by inc). field_a is stored but not indexed due to the large number of distinct values it can have. I need to index field_b (I need facet and stats on it) which is not in the document but its value is based on a calculation of the recent (e.g. summed) value of field_a. There is no way to do it nowadays. So I thought of a new method: custom atomic update. There will be a new interface in Solr: public interface CustomAtomicUpdater { public void update(SolrInputDocument oldDoc, String fieldName, Object fieldVal) ; } There will be a new attribute for fields in schema.xml called customAtomicUpdateClass (and all support in code, of course). The value is a class which is an implementation of CustomAtomicUpdater. In our example it will be defined for field_a. In method getUpdatedDocument in DistributedUpdateProcessor.java, we will add handling of custom case: } else if (custom.equals(key)) { updateField = true; SchemaField sf = schema.getField(sif.getName()); String customAtomicUpdaterClassName = sf.getCustomAtomicUpdaterClass(); if (customAtomicUpdaterClassName == null) { throw new SolrException(ErrorCode.BAD_REQUEST, There is no customAtomicUpdaterClass defined for + sif + .); } CustomAtomicUpdater updater = schema.getResourceLoader() .newInstance(customAtomicUpdaterClassName, CustomAtomicUpdater.class); if (updater == null) { throw new SolrException(ErrorCode.BAD_REQUEST, Was unable to create instance of + customAtomicUpdaterClassName + .); } updater.update(oldDoc, sif.getName(), fieldVal); } In my implementation I will sum field_a (oldvalue + newvalue) and update field_b according to my logic. Example of use: add doc field name=field_a update=custom128/field /doc /add What do say about my suggestion? Thanks. Email secured by Check Point Email secured by Check Point
Re: suggestion for new custom atomic update
You can get the summed value, 13, if you add a processor after DistributedUpdateProcessorFactory in the URP chain. Then one possibility would be to clone this value to another field, such as field_b, and run other processors on that field. Or for something more customized, you can use the StatelessScriptUpdateProcessorFactory, and retrieve the value of field_a with: var doc = cmd.solrDoc; // org.apache.solr.common.SolrInputDocument var field_a = doc.getFieldValue(field_a); Note if you try to get the value of field_a before DistributedUpdateProcessorFactory, then using your example with an atomic update, the value would be 5 (the value of the increment from the input document). On 27 October 2014 18:03, Elran Dvir elr...@checkpoint.com wrote: I will explain with an example. Let's say field_a is sent in the update with the value of 5. field_a is already stored in the document with the value 8. After the update field_a should have the value 13 (sum). The value of field_b will be based on the value of 13 and not 5. Is there a way in URP to know what is the value which is already stored in field_a? Thank you very much. -Original Message- From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] Sent: Sunday, October 26, 2014 6:07 PM To: solr-user Subject: Re: suggestion for new custom atomic update I am not sure what the problem is. URP catches all operations. So, you can modify the source document to add the calculation when the field_a is either new or updated. Or are you trying to calculate things across multiple documents? In that case, neither mine nor your solution will work, I think. Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 On 26 October 2014 12:00, Elran Dvir elr...@checkpoint.com wrote: Thanks for your response. If the calculation is based on the most recent summed value of field_a and the value of field_a in the update, how can I? Thanks. -Original Message- From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] Sent: Sunday, October 26, 2014 2:11 PM To: solr-user Subject: RE: suggestion for new custom atomic update Can't you do the calculation in custom UpdateRequestProcessor? Regards, Alex On 26/10/2014 4:17 am, Elran Dvir elr...@checkpoint.com wrote: Hi all, Did anyone have a chance to review my idea? Thanks. -Original Message- From: Elran Dvir Sent: Monday, October 20, 2014 12:42 PM To: solr-user Subject: suggestion for new custom atomic update Hi all, This is my use case: I have a stored field, field_a, which is atomic updated (let's say by inc). field_a is stored but not indexed due to the large number of distinct values it can have. I need to index field_b (I need facet and stats on it) which is not in the document but its value is based on a calculation of the recent (e.g. summed) value of field_a. There is no way to do it nowadays. So I thought of a new method: custom atomic update. There will be a new interface in Solr: public interface CustomAtomicUpdater { public void update(SolrInputDocument oldDoc, String fieldName, Object fieldVal) ; } There will be a new attribute for fields in schema.xml called customAtomicUpdateClass (and all support in code, of course). The value is a class which is an implementation of CustomAtomicUpdater. In our example it will be defined for field_a. In method getUpdatedDocument in DistributedUpdateProcessor.java, we will add handling of custom case: } else if (custom.equals(key)) { updateField = true; SchemaField sf = schema.getField(sif.getName()); String customAtomicUpdaterClassName = sf.getCustomAtomicUpdaterClass(); if (customAtomicUpdaterClassName == null) { throw new SolrException(ErrorCode.BAD_REQUEST, There is no customAtomicUpdaterClass defined for + sif + .); } CustomAtomicUpdater updater = schema.getResourceLoader() .newInstance(customAtomicUpdaterClassName, CustomAtomicUpdater.class); if (updater == null) { throw new SolrException(ErrorCode.BAD_REQUEST, Was unable to create instance of + customAtomicUpdaterClassName + .); } updater.update(oldDoc, sif.getName(), fieldVal); } In my implementation I will sum field_a (oldvalue + newvalue) and update field_b according to my logic. Example of use: add doc field name=field_a update=custom128/field /doc /add What do say about my suggestion? Thanks. Email secured by Check Point Email secured by Check Point
RE: suggestion for new custom atomic update
Thank you very much for your suggestion. I created an update processor factory with my logic. I changed the update processor chain to be: processor class=solr.LogUpdateProcessorFactory / processor class=solr.RunUpdateProcessorFactory / processor class=mycode.solr_plugins.FieldManipulationProcessorFactory / But nothing seems to happen. When I move my class to be the first in the chain, the logic is running (not as I want, of course. It's calculated based on the update value rather than the stored value) . How can I define a custom update processor factory that will run after DistributedUpdateProcessorFactory? Thank you very much. -Original Message- From: Matthew Nigl [mailto:matthew.n...@gmail.com] Sent: Monday, October 27, 2014 12:10 PM To: solr-user@lucene.apache.org Subject: Re: suggestion for new custom atomic update You can get the summed value, 13, if you add a processor after DistributedUpdateProcessorFactory in the URP chain. Then one possibility would be to clone this value to another field, such as field_b, and run other processors on that field. Or for something more customized, you can use the StatelessScriptUpdateProcessorFactory, and retrieve the value of field_a with: var doc = cmd.solrDoc; // org.apache.solr.common.SolrInputDocument var field_a = doc.getFieldValue(field_a); Note if you try to get the value of field_a before DistributedUpdateProcessorFactory, then using your example with an atomic update, the value would be 5 (the value of the increment from the input document). On 27 October 2014 18:03, Elran Dvir elr...@checkpoint.com wrote: I will explain with an example. Let's say field_a is sent in the update with the value of 5. field_a is already stored in the document with the value 8. After the update field_a should have the value 13 (sum). The value of field_b will be based on the value of 13 and not 5. Is there a way in URP to know what is the value which is already stored in field_a? Thank you very much. -Original Message- From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] Sent: Sunday, October 26, 2014 6:07 PM To: solr-user Subject: Re: suggestion for new custom atomic update I am not sure what the problem is. URP catches all operations. So, you can modify the source document to add the calculation when the field_a is either new or updated. Or are you trying to calculate things across multiple documents? In that case, neither mine nor your solution will work, I think. Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 On 26 October 2014 12:00, Elran Dvir elr...@checkpoint.com wrote: Thanks for your response. If the calculation is based on the most recent summed value of field_a and the value of field_a in the update, how can I? Thanks. -Original Message- From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] Sent: Sunday, October 26, 2014 2:11 PM To: solr-user Subject: RE: suggestion for new custom atomic update Can't you do the calculation in custom UpdateRequestProcessor? Regards, Alex On 26/10/2014 4:17 am, Elran Dvir elr...@checkpoint.com wrote: Hi all, Did anyone have a chance to review my idea? Thanks. -Original Message- From: Elran Dvir Sent: Monday, October 20, 2014 12:42 PM To: solr-user Subject: suggestion for new custom atomic update Hi all, This is my use case: I have a stored field, field_a, which is atomic updated (let's say by inc). field_a is stored but not indexed due to the large number of distinct values it can have. I need to index field_b (I need facet and stats on it) which is not in the document but its value is based on a calculation of the recent (e.g. summed) value of field_a. There is no way to do it nowadays. So I thought of a new method: custom atomic update. There will be a new interface in Solr: public interface CustomAtomicUpdater { public void update(SolrInputDocument oldDoc, String fieldName, Object fieldVal) ; } There will be a new attribute for fields in schema.xml called customAtomicUpdateClass (and all support in code, of course). The value is a class which is an implementation of CustomAtomicUpdater. In our example it will be defined for field_a. In method getUpdatedDocument in DistributedUpdateProcessor.java, we will add handling of custom case: } else if (custom.equals(key)) { updateField = true; SchemaField sf = schema.getField(sif.getName()); String customAtomicUpdaterClassName = sf.getCustomAtomicUpdaterClass(); if (customAtomicUpdaterClassName == null) { throw new SolrException(ErrorCode.BAD_REQUEST, There is no customAtomicUpdaterClass defined
Re: suggestion for new custom atomic update
Hi Elran, You need to explicitly specify the DistributedUpdateProcessorFactory in the chain and then add your custom processor after it. On Mon, Oct 27, 2014 at 9:26 PM, Elran Dvir elr...@checkpoint.com wrote: Thank you very much for your suggestion. I created an update processor factory with my logic. I changed the update processor chain to be: processor class=solr.LogUpdateProcessorFactory / processor class=solr.RunUpdateProcessorFactory / processor class=mycode.solr_plugins.FieldManipulationProcessorFactory / But nothing seems to happen. When I move my class to be the first in the chain, the logic is running (not as I want, of course. It's calculated based on the update value rather than the stored value) . How can I define a custom update processor factory that will run after DistributedUpdateProcessorFactory? Thank you very much. -Original Message- From: Matthew Nigl [mailto:matthew.n...@gmail.com] Sent: Monday, October 27, 2014 12:10 PM To: solr-user@lucene.apache.org Subject: Re: suggestion for new custom atomic update You can get the summed value, 13, if you add a processor after DistributedUpdateProcessorFactory in the URP chain. Then one possibility would be to clone this value to another field, such as field_b, and run other processors on that field. Or for something more customized, you can use the StatelessScriptUpdateProcessorFactory, and retrieve the value of field_a with: var doc = cmd.solrDoc; // org.apache.solr.common.SolrInputDocument var field_a = doc.getFieldValue(field_a); Note if you try to get the value of field_a before DistributedUpdateProcessorFactory, then using your example with an atomic update, the value would be 5 (the value of the increment from the input document). On 27 October 2014 18:03, Elran Dvir elr...@checkpoint.com wrote: I will explain with an example. Let's say field_a is sent in the update with the value of 5. field_a is already stored in the document with the value 8. After the update field_a should have the value 13 (sum). The value of field_b will be based on the value of 13 and not 5. Is there a way in URP to know what is the value which is already stored in field_a? Thank you very much. -Original Message- From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] Sent: Sunday, October 26, 2014 6:07 PM To: solr-user Subject: Re: suggestion for new custom atomic update I am not sure what the problem is. URP catches all operations. So, you can modify the source document to add the calculation when the field_a is either new or updated. Or are you trying to calculate things across multiple documents? In that case, neither mine nor your solution will work, I think. Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 On 26 October 2014 12:00, Elran Dvir elr...@checkpoint.com wrote: Thanks for your response. If the calculation is based on the most recent summed value of field_a and the value of field_a in the update, how can I? Thanks. -Original Message- From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] Sent: Sunday, October 26, 2014 2:11 PM To: solr-user Subject: RE: suggestion for new custom atomic update Can't you do the calculation in custom UpdateRequestProcessor? Regards, Alex On 26/10/2014 4:17 am, Elran Dvir elr...@checkpoint.com wrote: Hi all, Did anyone have a chance to review my idea? Thanks. -Original Message- From: Elran Dvir Sent: Monday, October 20, 2014 12:42 PM To: solr-user Subject: suggestion for new custom atomic update Hi all, This is my use case: I have a stored field, field_a, which is atomic updated (let's say by inc). field_a is stored but not indexed due to the large number of distinct values it can have. I need to index field_b (I need facet and stats on it) which is not in the document but its value is based on a calculation of the recent (e.g. summed) value of field_a. There is no way to do it nowadays. So I thought of a new method: custom atomic update. There will be a new interface in Solr: public interface CustomAtomicUpdater { public void update(SolrInputDocument oldDoc, String fieldName, Object fieldVal) ; } There will be a new attribute for fields in schema.xml called customAtomicUpdateClass (and all support in code, of course). The value is a class which is an implementation of CustomAtomicUpdater. In our example it will be defined for field_a. In method getUpdatedDocument in DistributedUpdateProcessor.java, we will add handling of custom case: } else if (custom.equals(key)) { updateField = true
Re: suggestion for new custom atomic update
No problem Elran. As Shalin mentioned, you will need to do it like this: processor class=solr.DistributedUpdateProcessorFactory/ processor class=mycode.solr_plugins.FieldManipulationProcessorFactory / processor class=solr.LogUpdateProcessorFactory / processor class=solr.RunUpdateProcessorFactory / On 28 October 2014 03:22, Shalin Shekhar Mangar shalinman...@gmail.com wrote: Hi Elran, You need to explicitly specify the DistributedUpdateProcessorFactory in the chain and then add your custom processor after it. On Mon, Oct 27, 2014 at 9:26 PM, Elran Dvir elr...@checkpoint.com wrote: Thank you very much for your suggestion. I created an update processor factory with my logic. I changed the update processor chain to be: processor class=solr.LogUpdateProcessorFactory / processor class=solr.RunUpdateProcessorFactory / processor class=mycode.solr_plugins.FieldManipulationProcessorFactory / But nothing seems to happen. When I move my class to be the first in the chain, the logic is running (not as I want, of course. It's calculated based on the update value rather than the stored value) . How can I define a custom update processor factory that will run after DistributedUpdateProcessorFactory? Thank you very much. -Original Message- From: Matthew Nigl [mailto:matthew.n...@gmail.com] Sent: Monday, October 27, 2014 12:10 PM To: solr-user@lucene.apache.org Subject: Re: suggestion for new custom atomic update You can get the summed value, 13, if you add a processor after DistributedUpdateProcessorFactory in the URP chain. Then one possibility would be to clone this value to another field, such as field_b, and run other processors on that field. Or for something more customized, you can use the StatelessScriptUpdateProcessorFactory, and retrieve the value of field_a with: var doc = cmd.solrDoc; // org.apache.solr.common.SolrInputDocument var field_a = doc.getFieldValue(field_a); Note if you try to get the value of field_a before DistributedUpdateProcessorFactory, then using your example with an atomic update, the value would be 5 (the value of the increment from the input document). On 27 October 2014 18:03, Elran Dvir elr...@checkpoint.com wrote: I will explain with an example. Let's say field_a is sent in the update with the value of 5. field_a is already stored in the document with the value 8. After the update field_a should have the value 13 (sum). The value of field_b will be based on the value of 13 and not 5. Is there a way in URP to know what is the value which is already stored in field_a? Thank you very much. -Original Message- From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] Sent: Sunday, October 26, 2014 6:07 PM To: solr-user Subject: Re: suggestion for new custom atomic update I am not sure what the problem is. URP catches all operations. So, you can modify the source document to add the calculation when the field_a is either new or updated. Or are you trying to calculate things across multiple documents? In that case, neither mine nor your solution will work, I think. Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 On 26 October 2014 12:00, Elran Dvir elr...@checkpoint.com wrote: Thanks for your response. If the calculation is based on the most recent summed value of field_a and the value of field_a in the update, how can I? Thanks. -Original Message- From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] Sent: Sunday, October 26, 2014 2:11 PM To: solr-user Subject: RE: suggestion for new custom atomic update Can't you do the calculation in custom UpdateRequestProcessor? Regards, Alex On 26/10/2014 4:17 am, Elran Dvir elr...@checkpoint.com wrote: Hi all, Did anyone have a chance to review my idea? Thanks. -Original Message- From: Elran Dvir Sent: Monday, October 20, 2014 12:42 PM To: solr-user Subject: suggestion for new custom atomic update Hi all, This is my use case: I have a stored field, field_a, which is atomic updated (let's say by inc). field_a is stored but not indexed due to the large number of distinct values it can have. I need to index field_b (I need facet and stats on it) which is not in the document but its value is based on a calculation of the recent (e.g. summed) value of field_a. There is no way to do it nowadays. So I thought of a new method: custom atomic update. There will be a new interface in Solr: public interface CustomAtomicUpdater { public void update(SolrInputDocument
RE: suggestion for new custom atomic update
Hi all, Did anyone have a chance to review my idea? Thanks. -Original Message- From: Elran Dvir Sent: Monday, October 20, 2014 12:42 PM To: solr-user Subject: suggestion for new custom atomic update Hi all, This is my use case: I have a stored field, field_a, which is atomic updated (let's say by inc). field_a is stored but not indexed due to the large number of distinct values it can have. I need to index field_b (I need facet and stats on it) which is not in the document but its value is based on a calculation of the recent (e.g. summed) value of field_a. There is no way to do it nowadays. So I thought of a new method: custom atomic update. There will be a new interface in Solr: public interface CustomAtomicUpdater { public void update(SolrInputDocument oldDoc, String fieldName, Object fieldVal) ; } There will be a new attribute for fields in schema.xml called customAtomicUpdateClass (and all support in code, of course). The value is a class which is an implementation of CustomAtomicUpdater. In our example it will be defined for field_a. In method getUpdatedDocument in DistributedUpdateProcessor.java, we will add handling of custom case: } else if (custom.equals(key)) { updateField = true; SchemaField sf = schema.getField(sif.getName()); String customAtomicUpdaterClassName = sf.getCustomAtomicUpdaterClass(); if (customAtomicUpdaterClassName == null) { throw new SolrException(ErrorCode.BAD_REQUEST, There is no customAtomicUpdaterClass defined for + sif + .); } CustomAtomicUpdater updater = schema.getResourceLoader() .newInstance(customAtomicUpdaterClassName, CustomAtomicUpdater.class); if (updater == null) { throw new SolrException(ErrorCode.BAD_REQUEST, Was unable to create instance of + customAtomicUpdaterClassName + .); } updater.update(oldDoc, sif.getName(), fieldVal); } In my implementation I will sum field_a (oldvalue + newvalue) and update field_b according to my logic. Example of use: add doc field name=field_a update=custom128/field /doc /add What do say about my suggestion? Thanks.
RE: suggestion for new custom atomic update
Can't you do the calculation in custom UpdateRequestProcessor? Regards, Alex On 26/10/2014 4:17 am, Elran Dvir elr...@checkpoint.com wrote: Hi all, Did anyone have a chance to review my idea? Thanks. -Original Message- From: Elran Dvir Sent: Monday, October 20, 2014 12:42 PM To: solr-user Subject: suggestion for new custom atomic update Hi all, This is my use case: I have a stored field, field_a, which is atomic updated (let's say by inc). field_a is stored but not indexed due to the large number of distinct values it can have. I need to index field_b (I need facet and stats on it) which is not in the document but its value is based on a calculation of the recent (e.g. summed) value of field_a. There is no way to do it nowadays. So I thought of a new method: custom atomic update. There will be a new interface in Solr: public interface CustomAtomicUpdater { public void update(SolrInputDocument oldDoc, String fieldName, Object fieldVal) ; } There will be a new attribute for fields in schema.xml called customAtomicUpdateClass (and all support in code, of course). The value is a class which is an implementation of CustomAtomicUpdater. In our example it will be defined for field_a. In method getUpdatedDocument in DistributedUpdateProcessor.java, we will add handling of custom case: } else if (custom.equals(key)) { updateField = true; SchemaField sf = schema.getField(sif.getName()); String customAtomicUpdaterClassName = sf.getCustomAtomicUpdaterClass(); if (customAtomicUpdaterClassName == null) { throw new SolrException(ErrorCode.BAD_REQUEST, There is no customAtomicUpdaterClass defined for + sif + .); } CustomAtomicUpdater updater = schema.getResourceLoader() .newInstance(customAtomicUpdaterClassName, CustomAtomicUpdater.class); if (updater == null) { throw new SolrException(ErrorCode.BAD_REQUEST, Was unable to create instance of + customAtomicUpdaterClassName + .); } updater.update(oldDoc, sif.getName(), fieldVal); } In my implementation I will sum field_a (oldvalue + newvalue) and update field_b according to my logic. Example of use: add doc field name=field_a update=custom128/field /doc /add What do say about my suggestion? Thanks.
RE: suggestion for new custom atomic update
Thanks for your response. If the calculation is based on the most recent summed value of field_a and the value of field_a in the update, how can I? Thanks. -Original Message- From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] Sent: Sunday, October 26, 2014 2:11 PM To: solr-user Subject: RE: suggestion for new custom atomic update Can't you do the calculation in custom UpdateRequestProcessor? Regards, Alex On 26/10/2014 4:17 am, Elran Dvir elr...@checkpoint.com wrote: Hi all, Did anyone have a chance to review my idea? Thanks. -Original Message- From: Elran Dvir Sent: Monday, October 20, 2014 12:42 PM To: solr-user Subject: suggestion for new custom atomic update Hi all, This is my use case: I have a stored field, field_a, which is atomic updated (let's say by inc). field_a is stored but not indexed due to the large number of distinct values it can have. I need to index field_b (I need facet and stats on it) which is not in the document but its value is based on a calculation of the recent (e.g. summed) value of field_a. There is no way to do it nowadays. So I thought of a new method: custom atomic update. There will be a new interface in Solr: public interface CustomAtomicUpdater { public void update(SolrInputDocument oldDoc, String fieldName, Object fieldVal) ; } There will be a new attribute for fields in schema.xml called customAtomicUpdateClass (and all support in code, of course). The value is a class which is an implementation of CustomAtomicUpdater. In our example it will be defined for field_a. In method getUpdatedDocument in DistributedUpdateProcessor.java, we will add handling of custom case: } else if (custom.equals(key)) { updateField = true; SchemaField sf = schema.getField(sif.getName()); String customAtomicUpdaterClassName = sf.getCustomAtomicUpdaterClass(); if (customAtomicUpdaterClassName == null) { throw new SolrException(ErrorCode.BAD_REQUEST, There is no customAtomicUpdaterClass defined for + sif + .); } CustomAtomicUpdater updater = schema.getResourceLoader() .newInstance(customAtomicUpdaterClassName, CustomAtomicUpdater.class); if (updater == null) { throw new SolrException(ErrorCode.BAD_REQUEST, Was unable to create instance of + customAtomicUpdaterClassName + .); } updater.update(oldDoc, sif.getName(), fieldVal); } In my implementation I will sum field_a (oldvalue + newvalue) and update field_b according to my logic. Example of use: add doc field name=field_a update=custom128/field /doc /add What do say about my suggestion? Thanks. Email secured by Check Point
Re: suggestion for new custom atomic update
I am not sure what the problem is. URP catches all operations. So, you can modify the source document to add the calculation when the field_a is either new or updated. Or are you trying to calculate things across multiple documents? In that case, neither mine nor your solution will work, I think. Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 On 26 October 2014 12:00, Elran Dvir elr...@checkpoint.com wrote: Thanks for your response. If the calculation is based on the most recent summed value of field_a and the value of field_a in the update, how can I? Thanks. -Original Message- From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] Sent: Sunday, October 26, 2014 2:11 PM To: solr-user Subject: RE: suggestion for new custom atomic update Can't you do the calculation in custom UpdateRequestProcessor? Regards, Alex On 26/10/2014 4:17 am, Elran Dvir elr...@checkpoint.com wrote: Hi all, Did anyone have a chance to review my idea? Thanks. -Original Message- From: Elran Dvir Sent: Monday, October 20, 2014 12:42 PM To: solr-user Subject: suggestion for new custom atomic update Hi all, This is my use case: I have a stored field, field_a, which is atomic updated (let's say by inc). field_a is stored but not indexed due to the large number of distinct values it can have. I need to index field_b (I need facet and stats on it) which is not in the document but its value is based on a calculation of the recent (e.g. summed) value of field_a. There is no way to do it nowadays. So I thought of a new method: custom atomic update. There will be a new interface in Solr: public interface CustomAtomicUpdater { public void update(SolrInputDocument oldDoc, String fieldName, Object fieldVal) ; } There will be a new attribute for fields in schema.xml called customAtomicUpdateClass (and all support in code, of course). The value is a class which is an implementation of CustomAtomicUpdater. In our example it will be defined for field_a. In method getUpdatedDocument in DistributedUpdateProcessor.java, we will add handling of custom case: } else if (custom.equals(key)) { updateField = true; SchemaField sf = schema.getField(sif.getName()); String customAtomicUpdaterClassName = sf.getCustomAtomicUpdaterClass(); if (customAtomicUpdaterClassName == null) { throw new SolrException(ErrorCode.BAD_REQUEST, There is no customAtomicUpdaterClass defined for + sif + .); } CustomAtomicUpdater updater = schema.getResourceLoader() .newInstance(customAtomicUpdaterClassName, CustomAtomicUpdater.class); if (updater == null) { throw new SolrException(ErrorCode.BAD_REQUEST, Was unable to create instance of + customAtomicUpdaterClassName + .); } updater.update(oldDoc, sif.getName(), fieldVal); } In my implementation I will sum field_a (oldvalue + newvalue) and update field_b according to my logic. Example of use: add doc field name=field_a update=custom128/field /doc /add What do say about my suggestion? Thanks. Email secured by Check Point
suggestion for new custom atomic update
Hi all, This is my use case: I have a stored field, field_a, which is atomic updated (let's say by inc). field_a is stored but not indexed due to the large number of distinct values it can have. I need to index field_b (I need facet and stats on it) which is not in the document but its value is based on a calculation of the recent (e.g. summed) value of field_a. There is no way to do it nowadays. So I thought of a new method: custom atomic update. There will be a new interface in Solr: public interface CustomAtomicUpdater { public void update(SolrInputDocument oldDoc, String fieldName, Object fieldVal) ; } There will be a new attribute for fields in schema.xml called customAtomicUpdateClass (and all support in code, of course). The value is a class which is an implementation of CustomAtomicUpdater. In our example it will be defined for field_a. In method getUpdatedDocument in DistributedUpdateProcessor.java, we will add handling of custom case: } else if (custom.equals(key)) { updateField = true; SchemaField sf = schema.getField(sif.getName()); String customAtomicUpdaterClassName = sf.getCustomAtomicUpdaterClass(); if (customAtomicUpdaterClassName == null) { throw new SolrException(ErrorCode.BAD_REQUEST, There is no customAtomicUpdaterClass defined for + sif + .); } CustomAtomicUpdater updater = schema.getResourceLoader() .newInstance(customAtomicUpdaterClassName, CustomAtomicUpdater.class); if (updater == null) { throw new SolrException(ErrorCode.BAD_REQUEST, Was unable to create instance of + customAtomicUpdaterClassName + .); } updater.update(oldDoc, sif.getName(), fieldVal); } In my implementation I will sum field_a (oldvalue + newvalue) and update field_b according to my logic. Example of use: add doc field name=field_a update=custom128/field /doc /add What do say about my suggestion? Thanks.