[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452586#comment-16452586 ] Maxence SAUNIER commented on CONNECTORS-1503: - So, I have test with an internal URP and I confirm, it working good. Thanks you very much Karl and Abe-san. > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Fix For: ManifoldCF 2.11 > > Attachments: 20170421-1740.png, CONNECTORS-1503.patch, > jira_update_processor.png, manifoldcf_arguments_uniqFields.png, > manifoldcf_output_conf.zip > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452549#comment-16452549 ] Maxence SAUNIER commented on CONNECTORS-1503: - YES ! It working !! Good job ! Thanks you very much. > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, CONNECTORS-1503.patch, > jira_update_processor.png, manifoldcf_arguments_uniqFields.png, > manifoldcf_output_conf.zip > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452302#comment-16452302 ] Maxence SAUNIER commented on CONNECTORS-1503: - Ok, I test to compile as soon as I have time. > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, CONNECTORS-1503.patch, > jira_update_processor.png, manifoldcf_arguments_uniqFields.png, > manifoldcf_output_conf.zip > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452290#comment-16452290 ] Karl Wright commented on CONNECTORS-1503: - [~Moltroon] I am willing to do this but I cannot until Sunday at the earliest. > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, CONNECTORS-1503.patch, > jira_update_processor.png, manifoldcf_arguments_uniqFields.png, > manifoldcf_output_conf.zip > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452284#comment-16452284 ] Karl Wright commented on CONNECTORS-1503: - r1830074 commits this tentative fix. > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, CONNECTORS-1503.patch, > jira_update_processor.png, manifoldcf_arguments_uniqFields.png, > manifoldcf_output_conf.zip > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452279#comment-16452279 ] Maxence SAUNIER commented on CONNECTORS-1503: - Can you compile me a binary and give me a link? Thanks, > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, CONNECTORS-1503.patch, > jira_update_processor.png, manifoldcf_arguments_uniqFields.png, > manifoldcf_output_conf.zip > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452269#comment-16452269 ] Karl Wright commented on CONNECTORS-1503: - [~Moltroon] I have attached a patch. Please deploy this and let me know if it works for you. Thanks!! > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, CONNECTORS-1503.patch, > jira_update_processor.png, manifoldcf_arguments_uniqFields.png, > manifoldcf_output_conf.zip > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452247#comment-16452247 ] Karl Wright commented on CONNECTORS-1503: - [~shinichiro abe] Ah, yes, that might work. I will code it up and attach a patch. > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, jira_update_processor.png, > manifoldcf_arguments_uniqFields.png, manifoldcf_output_conf.zip > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[
https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452238#comment-16452238
]
Shinichiro Abe commented on CONNECTORS-1503:
Maybe we have to break Solr#add(doc) down to SolrRequest#process(SolrClient
client, String collection) as well as contentStreamUpdateRequest.process(
solrServer ) at useExtractUpdateHandler=true.
{noformat}
//response = solrServer.add( currentSolrDoc ); // <- current
impl
UpdateRequest req = new UpdateRequest();
req.setParams(params); // <- ModifiableSolrParams: params
through writeField(...)
req.add(currentSolrDoc); // <- SolrInputDocument
req.setCommitWithin(commitWithinMs);
response = req.process(solrServer, (String)null); // <-
default collection
{noformat}
> UpdateProcessor SolrCloud and ManifoldCF
>
>
> Key: CONNECTORS-1503
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1503
> Project: ManifoldCF
> Issue Type: Bug
> Components: Solr 6.x component
>Affects Versions: ManifoldCF 2.9.1
> Environment: SolrCloud 6.6
> ManifoldCF 2.9.1
>Reporter: Maxence SAUNIER
>Assignee: Shinichiro Abe
>Priority: Major
> Attachments: 20170421-1740.png, jira_update_processor.png,
> manifoldcf_arguments_uniqFields.png, manifoldcf_output_conf.zip
>
>
> Hello,
> [Link to Apache mail
> archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E]
> When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they
> arguments on the POST request and not on the url parameters. So, for add a
> (pre)processor or a post-processor with the url, it's not possible.
> [SolrConfig
> updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_]
> [call
> UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters]
> [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png]
> Solr response:
> org.apache.solr.common.SolrException: ERROR:
> [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc]
> unknown field 'processor'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452200#comment-16452200 ] Karl Wright commented on CONNECTORS-1503: - [~shinichiro abe] I am still confused as to where this should happen. We cannot use writeField(ModifiableSolrParams out, String fieldName, String fieldValue) when building a SolrInputDocument, because in that case SolrJ is building the request entirely, not ManifoldCF. > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, jira_update_processor.png, > manifoldcf_arguments_uniqFields.png, manifoldcf_output_conf.zip > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452195#comment-16452195 ] Karl Wright commented on CONNECTORS-1503: - [~Moltroon], for Solr Cloud that will not work. > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, jira_update_processor.png, > manifoldcf_arguments_uniqFields.png, manifoldcf_output_conf.zip > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452143#comment-16452143 ] Shinichiro Abe commented on CONNECTORS-1503: Yes, you are right. it should be writeField(ModifiableSolrParams out, String fieldName, String fieldValue) I was using standard handler with curl-based postings, which is not using MCF' tika option. > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, jira_update_processor.png, > manifoldcf_arguments_uniqFields.png, manifoldcf_output_conf.zip > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[
https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452135#comment-16452135
]
Maxence SAUNIER commented on CONNECTORS-1503:
-
For add a precision, you can have processor or post-processor on the HttpPost
argument. And you can have many URP for the same arguments.
Solr example with curl:
curl
"http://localhost:8983/solr/gettingstarted/update/json?processor=remove_blanks,uniqField=signature=true;
-H 'Content-type: application/json' -d '
[
{
"name" : "The Lightning Thief",
"features" : "This is just a test",
"cat" : ["book","hardcover"]
},
{
"name" : "The Lightning Thief",
"features" : "This is just a test",
"cat" : ["book","hardcover"]
}
]'
> UpdateProcessor SolrCloud and ManifoldCF
>
>
> Key: CONNECTORS-1503
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1503
> Project: ManifoldCF
> Issue Type: Bug
> Components: Solr 6.x component
>Affects Versions: ManifoldCF 2.9.1
> Environment: SolrCloud 6.6
> ManifoldCF 2.9.1
>Reporter: Maxence SAUNIER
>Assignee: Shinichiro Abe
>Priority: Major
> Attachments: 20170421-1740.png, jira_update_processor.png,
> manifoldcf_arguments_uniqFields.png, manifoldcf_output_conf.zip
>
>
> Hello,
> [Link to Apache mail
> archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E]
> When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they
> arguments on the POST request and not on the url parameters. So, for add a
> (pre)processor or a post-processor with the url, it's not possible.
> [SolrConfig
> updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_]
> [call
> UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters]
> [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png]
> Solr response:
> org.apache.solr.common.SolrException: ERROR:
> [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc]
> unknown field 'processor'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[
https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452086#comment-16452086
]
Karl Wright commented on CONNECTORS-1503:
-
[~shinichiro abe], you have confirmed that if you set MCF Solr Connection to
not use extracting update handler, and you set an argument
"processor=", it properly obeys the processor argument?
The reason I wonder about this is because the code in HttpPoster for
SolrInputDocument does not distinguish between fields and arguments: it uses
addField() for both, e.g.:
{code}
if (contentAttributeName != null)
{
// Copy the content into a string. This is a bad thing to do, but we
have no choice given SolrJ architecture at this time.
// We enforce a size limit upstream.
Reader r = new InputStreamReader(is, Consts.UTF_8);
StringBuilder sb = new StringBuilder((int)length);
char[] buffer = new char[65536];
while (true)
{
int amt = r.read(buffer,0,buffer.length);
if (amt == -1)
break;
sb.append(buffer,0,amt);
}
outputDoc.addField( contentAttributeName, sb.toString() );
}
...
// Write the arguments
for ( String name : arguments.keySet() )
{
List values = arguments.get( name );
outputDoc.addField( name, values );
}
...
{code}
I am pretty sure that fields and arguments would need to be handled
differently, no?
> UpdateProcessor SolrCloud and ManifoldCF
>
>
> Key: CONNECTORS-1503
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1503
> Project: ManifoldCF
> Issue Type: Bug
> Components: Solr 6.x component
>Affects Versions: ManifoldCF 2.9.1
> Environment: SolrCloud 6.6
> ManifoldCF 2.9.1
>Reporter: Maxence SAUNIER
>Assignee: Shinichiro Abe
>Priority: Major
> Attachments: 20170421-1740.png, jira_update_processor.png,
> manifoldcf_arguments_uniqFields.png, manifoldcf_output_conf.zip
>
>
> Hello,
> [Link to Apache mail
> archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E]
> When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they
> arguments on the POST request and not on the url parameters. So, for add a
> (pre)processor or a post-processor with the url, it's not possible.
> [SolrConfig
> updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_]
> [call
> UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters]
> [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png]
> Solr response:
> org.apache.solr.common.SolrException: ERROR:
> [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc]
> unknown field 'processor'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452080#comment-16452080 ] Shinichiro Abe commented on CONNECTORS-1503: in my env., standard handler with processor param works well, something is wrong in your env., imo. solr processor works in each node per doc, i do not think solr client have a ploblem. also, solr cell has solrcontenthandler that captures content body correctory, otoh mcf' tika extractor does not have it. thare is a difference. i want verbosed exeption stacktrace. > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, jira_update_processor.png, > manifoldcf_arguments_uniqFields.png, manifoldcf_output_conf.zip > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452062#comment-16452062 ] Karl Wright commented on CONNECTORS-1503: - [~shinichiro abe], I am trying to figure out how you are supposed to use non-field arguments with the standard SolrClient.add(SolrInputDocument) type of SolrJ request. It does not look like you can add these arguments into the SolrInputDocument itself and have it work. So that means these must be added to the SolrClient as part of setting it up. We construct the SolrClient in HttpPoster in one of two ways: either by building this ourselves, or by using the CloudSolrClientBuilder. We could try adding the arguments to the URL ourselves in the single-server case, but how would this be done in the CloudSolrClientBuilder? How are we supposed to do this? > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, jira_update_processor.png, > manifoldcf_arguments_uniqFields.png, manifoldcf_output_conf.zip > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452053#comment-16452053 ] Karl Wright commented on CONNECTORS-1503: - [~Moltroon], I have reviewed the code. In the Extracting Update Handler case, the Solr Connector builds a request itself. All fields in the request have a "literal" field specifier added, but the arguments specified in the connection do not, so clearly there is a difference in how these are presented to Solr. But in the standard Update Handler case, all we can do with SolrJ is build a SolrInputDocument and call the .add() method with it, and there is no way to specify any difference between fields and arguments. This is why you are having problems using "processor=" with the standard Update Handler. We are limited here by what SolrJ is capable of, but I cannot believe there is no way to tell Solr to use the processor argument. I will see if I can figure out how SolrJ expects this to happen in the standard update handler case. > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, jira_update_processor.png, > manifoldcf_arguments_uniqFields.png, manifoldcf_output_conf.zip > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452048#comment-16452048 ] Maxence SAUNIER commented on CONNECTORS-1503: - I think you have understood. I not used Solr Cell and Solr Cell is called by the Extracting Update Handler. My process to crawl: (1) I crawl files with ManifoldCF and ManifoldCF extract Metadata and send in json to Solr (2) Param "processor" work on solr to format data and register just the result on my collection With Extracting Update Handler, I think ManifoldCF send the file to Solr and Solr extract they metadata, it's that? I think with the standard Update Handler, ManifoldCF send processor on the JSON result, is correct? So, I just used the standard Update Request Handler to indexed my documents today. I do not do the right thing? Or is it a feature to consider in ManifoldCF? > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, jira_update_processor.png, > manifoldcf_arguments_uniqFields.png, manifoldcf_output_conf.zip > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452026#comment-16452026 ] Karl Wright commented on CONNECTORS-1503: - Hi [~Moltroon], I'm still just trying to understand how you have things set up. (1) The parameters you use to configure the Tika extractor are not affected by how you configure the Solr output connector. (2) It sounds like you have a complex pipeline, which probably includes the Tika Extractor, and the Metadata Adjuster too. It sounds furthermore like the Metadata Adjuster comes after the Tika Extractor. Is this in fact the case? (3) If that is your setup, then the right handler to use is not the Extracting Update Handler. It is the standard Update Handler. (4) It sounds like the "processor" argument works with the Extracting Update Handler but does not work with the standard Update Handler. Is this summary correct? If it is, I can look into why the processor argument is not working for that handler. Please verify. > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, jira_update_processor.png, > manifoldcf_arguments_uniqFields.png, manifoldcf_output_conf.zip > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[
https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451996#comment-16451996
]
Maxence SAUNIER commented on CONNECTORS-1503:
-
Hello Karl,
I did some tests today and the problem persist. I don't know if the problem is
ManifoldCF or Solr config.
Explanation:
If I check used Request Handler on ManifoldCF, I have
org.apache.tika.exception.ZeroByteFileException: InputStream must have > 0
bytes. I have files with no content but if I uncheck Request Handler on
ManifoldCF Tika ignore this problem dans just not send content field. Content
field not required on my Solr Schema.
So, I have modify the /update/extract requestHandler to unactive parameters but
not solved my problem.
And, if I uncheck used Request Handler on ManifoldCF, my content is just the
content file. Without the checkbox, content = "{file_lenght} {mime_type}
{other} {content_text}"
Tika is on the ManifoldCF and is a Tika exception, possible Tika parameters are
differents if I check used Request Handler? Or I have an default update
processor on Solr and this have a problem?
Thanks
> UpdateProcessor SolrCloud and ManifoldCF
>
>
> Key: CONNECTORS-1503
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1503
> Project: ManifoldCF
> Issue Type: Bug
> Components: Solr 6.x component
>Affects Versions: ManifoldCF 2.9.1
> Environment: SolrCloud 6.6
> ManifoldCF 2.9.1
>Reporter: Maxence SAUNIER
>Assignee: Shinichiro Abe
>Priority: Major
> Attachments: 20170421-1740.png, jira_update_processor.png,
> manifoldcf_arguments_uniqFields.png, manifoldcf_output_conf.zip
>
>
> Hello,
> [Link to Apache mail
> archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E]
> When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they
> arguments on the POST request and not on the url parameters. So, for add a
> (pre)processor or a post-processor with the url, it's not possible.
> [SolrConfig
> updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_]
> [call
> UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters]
> [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png]
> Solr response:
> org.apache.solr.common.SolrException: ERROR:
> [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc]
> unknown field 'processor'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450313#comment-16450313 ] Karl Wright commented on CONNECTORS-1503: - Hi [~Moltroon], This exception comes from Tika, and clearly also from the Tika running in Solr, as you know. You can disable the exception, as you know. But if your documents have no content, what are you indexing? Are you trying to index metadata only? Or do you expect there to be content but there isn't any getting sent to Solr? You can perhaps learn more about what's being sent to Solr by looking at the Solr log [INFO] messages -- which should tell you the content length (among other things). If you are seeing a zero content length there, then something is wrong in how you have set up your pipeline in ManifoldCF. If the content length is *not* zero, then something is wrong with how you have set up Solr. > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, jira_update_processor.png, > manifoldcf_arguments_uniqFields.png, manifoldcf_output_conf.zip > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450059#comment-16450059 ] Maxence SAUNIER commented on CONNECTORS-1503: - I have find the problem. I have checked "Use the Extract Update Handler:" param and URP working BUT If I check this parameter, I lost content value and Solr says: null:org.apache.solr.common.SolrException: org.apache.tika.exception.ZeroByteFileException: InputStream must have > 0 bytes If I ignore tika exception, my URP work, my documents are indexed BUT not have content field on Solr. Abe-san, do you know this problem? Thanks, > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, jira_update_processor.png, > manifoldcf_arguments_uniqFields.png, manifoldcf_output_conf.zip > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450031#comment-16450031 ] Karl Wright commented on CONNECTORS-1503: - [~Moltroon], I don't think there's been any changes of significance to the Solr connector in 2.9.1 or 2.10. > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, jira_update_processor.png, > manifoldcf_arguments_uniqFields.png > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450015#comment-16450015 ] Karl Wright commented on CONNECTORS-1503: - Hi [~Moltroon], can you attach a screen shot of the view page for your Solr connection? Also, [~shinichiro abe], can you do the same, so that we might compare? Thanks! > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, jira_update_processor.png, > manifoldcf_arguments_uniqFields.png > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450011#comment-16450011 ] Maxence SAUNIER commented on CONNECTORS-1503: - hello Karl, Hello Abe-san, I have test on Solr 7.3 (7.2.1 not available today) I have the same problem. I don't understand. HttpPost work but not arguments on ManifoldCF. Thanks, Maxence, > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, jira_update_processor.png, > manifoldcf_arguments_uniqFields.png > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448158#comment-16448158 ] Maxence SAUNIER commented on CONNECTORS-1503: - I could try tomorrow. Thanks, > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, jira_update_processor.png, > manifoldcf_arguments_uniqFields.png > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447927#comment-16447927 ] Karl Wright commented on CONNECTORS-1503: - SolrJ is supposed to be backwards-compatible with earlier versions of Solr, so *supposedly* that is not a concern. But I could imagine a bug in SolrJ, or in Solr 6.x, that might make it not work. [~Moltroon], I guess the next step is to try the unique fields test against Solr 7.x and see if that works for you. > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, jira_update_processor.png, > manifoldcf_arguments_uniqFields.png > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447908#comment-16447908 ] Shinichiro Abe commented on CONNECTORS-1503: 1. the "unique fields" processor test with ManifoldCF itself works well. 2. curl-based HTTPPost request with "processor=" works with Solr. I don't have any exception with ManifoldCF/Solr. Btw, I'm using Solr 7.2 since ManifoldCF's SolrJ version is 7x. > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, jira_update_processor.png, > manifoldcf_arguments_uniqFields.png > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447833#comment-16447833 ] Karl Wright commented on CONNECTORS-1503: - So let me summarize. It sounds like [~shinichiro abe] was able to get the unique value processor to work via http post under curl, correct? But [~Moltroon] was not able to get it to work in ManifoldCF? The things we need to do are: (1) Abe-san needs to try to set up the "unique fields" processor test with ManifoldCF itself. Does this work, or not? (2) If not, can we verify that a curl-based HTTPPost request with "processor=" works with Solr, or not? With that information, it should be possible to determine whether the issue lies with ManifoldCF or SolrJ. > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, jira_update_processor.png, > manifoldcf_arguments_uniqFields.png > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[
https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447764#comment-16447764
]
Maxence SAUNIER commented on CONNECTORS-1503:
-
Hello Karl, Hello Abe-san,
Sorry for the time to respond. I was busy this weekend.
So, I have test and I have this result :
*1. Test with Manifold and uniqFields :*
(attached my config)
With ManifodCF, literal.f1_ss = ["a", "a"]. So, URP not working.
*2. With http request*
{code:java}
curl "http://myserver:8983/solr/dev1/update?processor=uniqFields=true;
-H 'Content-type: application/json' -d '[{"literal.f1_ss": ["a", "a"]
, "id":"test"}]'
{code}
literal.f1_ss = ["a"]
So, the duplicate value is deleted. URP work.
> UpdateProcessor SolrCloud and ManifoldCF
>
>
> Key: CONNECTORS-1503
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1503
> Project: ManifoldCF
> Issue Type: Bug
> Components: Solr 6.x component
>Affects Versions: ManifoldCF 2.9.1
> Environment: SolrCloud 6.6
> ManifoldCF 2.9.1
>Reporter: Maxence SAUNIER
>Assignee: Shinichiro Abe
>Priority: Major
> Attachments: 20170421-1740.png, jira_update_processor.png,
> manifoldcf_arguments_uniqFields.png
>
>
> Hello,
> [Link to Apache mail
> archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E]
> When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they
> arguments on the POST request and not on the url parameters. So, for add a
> (pre)processor or a post-processor with the url, it's not possible.
> [SolrConfig
> updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_]
> [call
> UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters]
> [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png]
> Solr response:
> org.apache.solr.common.SolrException: ERROR:
> [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc]
> unknown field 'processor'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446762#comment-16446762 ] Karl Wright commented on CONNECTORS-1503: - Thank you, Abe-san. [~Moltroon], can you once again repeat Abe-san's steps, and see if you see the same results? > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, jira_update_processor.png > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446681#comment-16446681 ] Shinichiro Abe commented on CONNECTORS-1503: attached my config, in that case, f1_ss field value is "a", which is unique value through Solr' URP. But when I remove that processor argument, f1_ss field value is ["a", "a"], it has two values. As far as I know, that URP works via HttpPoster. > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: 20170421-1740.png, jira_update_processor.png > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446660#comment-16446660 ] Karl Wright commented on CONNECTORS-1503: - [~shinichiro abe], it seems like not getting an error from Solr is not enough, if it doesn't work either. Are you able to get a "processor" selection via the ManifoldCF Solr Output Connector to work in Solr? > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: jira_update_processor.png > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446415#comment-16446415 ] Maxence SAUNIER commented on CONNECTORS-1503: - Hi Karl, No problem. I can adapt my availability. Thanks, > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: jira_update_processor.png > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446083#comment-16446083 ] Karl Wright commented on CONNECTORS-1503: - Hi Maxence, Abe-san is in Tokyo and it will be a while before he is awake. Sorry! > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: jira_update_processor.png > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16445573#comment-16445573 ] Maxence SAUNIER commented on CONNECTORS-1503: - I have reproduce your example and same result: "processor":["uniqFields"] filed is create but porcessor not execute ManifoldCF output config: !jira_update_processor.png! How can I help you to debug the problem ? Thanks you. > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > Attachments: jira_update_processor.png > > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[
https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16445492#comment-16445492
]
Maxence SAUNIER commented on CONNECTORS-1503:
-
Hello Shinichiro, Hello Karl,
[~kwri...@metacarta.com] "can you take a fresh solr instance and try Abe-san's
experiment with it? "
Yes, I will test today this example.
I bring some clarifications.
*My need:*
I use an update processor personalized call *CityaTestUpdateProcessor*
This UpdateProcessor, update many field in Solr, so I add this Update processor
in the default UpdateProcessorChain.
My *configoverlay.json*:
{code:json}
{
"runtimeLib":{"CityaTestUpdateProcessorJar":{
"name":"CityaTestUpdateProcessorJar",
"version":9}},
"updateProcessor":{"CityaTestUpdateProcessorJar":{
"name":"CityaTestUpdateProcessorJar",
"class":"com.citya.CityaTestUpdateProcessorFactory",
"runtimeLib":"true",
"version":"9"}}
}
{code}
With this code, it's *work good*:
{code}
curl
"http://srv-formation-solr:8983/solr/dev1/update?*processor=CityaTestUpdateProcessorJar*=true;
-H 'Content-type: application/json' -d '[{"content": "test.test",
"id":"file:/srvics35/ways_montauban/gestion_ged/gerance/1129/1129003700059599/_BAUX_AVENANTS/60413_LOC_MANDAT_PRLVLMT_001.PDF"}]'
{code}
I test your example and I make you a return.
Thanks you,
> UpdateProcessor SolrCloud and ManifoldCF
>
>
> Key: CONNECTORS-1503
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1503
> Project: ManifoldCF
> Issue Type: Bug
> Components: Solr 6.x component
>Affects Versions: ManifoldCF 2.9.1
> Environment: SolrCloud 6.6
> ManifoldCF 2.9.1
>Reporter: Maxence SAUNIER
>Assignee: Shinichiro Abe
>Priority: Major
>
> Hello,
> [Link to Apache mail
> archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E]
> When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they
> arguments on the POST request and not on the url parameters. So, for add a
> (pre)processor or a post-processor with the url, it's not possible.
> [SolrConfig
> updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_]
> [call
> UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters]
> [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png]
> Solr response:
> org.apache.solr.common.SolrException: ERROR:
> [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc]
> unknown field 'processor'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16444940#comment-16444940 ] Karl Wright commented on CONNECTORS-1503: - [~shinichiro abe], thank you for looking into this. [~Moltroon], can you take a fresh solr instance and try Abe-san's experiment with it? If it works for you too, then maybe we can work to figure out what is different between that setup and the one where you get the failure. > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[
https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16444622#comment-16444622
]
Shinichiro Abe commented on CONNECTORS-1503:
I could not reproduce it.
{noformat}
curl http://localhost:8983/solr/collection1/config -d '{
"add-updateprocessor" : {
"name": "uniqFields",
"class":"solr.UniqFieldsUpdateProcessorFactory",
"fieldName":"content"
}
}'
{noformat}
And put processor=uniqFields on Auguments tab in MCF.
I didn't get any Solr error response, it seems to work.
it doesn't matter for methods, GET or POST.
> UpdateProcessor SolrCloud and ManifoldCF
>
>
> Key: CONNECTORS-1503
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1503
> Project: ManifoldCF
> Issue Type: Bug
> Components: Solr 6.x component
>Affects Versions: ManifoldCF 2.9.1
> Environment: SolrCloud 6.6
> ManifoldCF 2.9.1
>Reporter: Maxence SAUNIER
>Assignee: Shinichiro Abe
>Priority: Major
>
> Hello,
> [Link to Apache mail
> archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E]
> When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they
> arguments on the POST request and not on the url parameters. So, for add a
> (pre)processor or a post-processor with the url, it's not possible.
> [SolrConfig
> updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_]
> [call
> UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters]
> [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png]
> Solr response:
> org.apache.solr.common.SolrException: ERROR:
> [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc]
> unknown field 'processor'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16444100#comment-16444100 ] Maxence SAUNIER commented on CONNECTORS-1503: - If you need precisions, I am available. > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CONNECTORS-1503) UpdateProcessor SolrCloud and ManifoldCF
[ https://issues.apache.org/jira/browse/CONNECTORS-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16444058#comment-16444058 ] Karl Wright commented on CONNECTORS-1503: - Hi [~shinichiro abe], can you figure out what we need to do here? I'm afraid I don't fully understand what the requirements are for sending this kind of information to Solr any more, and how you are supposed to do it with SolrJ. I'm happy to implement it if I know how. > UpdateProcessor SolrCloud and ManifoldCF > > > Key: CONNECTORS-1503 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1503 > Project: ManifoldCF > Issue Type: Bug > Components: Solr 6.x component >Affects Versions: ManifoldCF 2.9.1 > Environment: SolrCloud 6.6 > ManifoldCF 2.9.1 >Reporter: Maxence SAUNIER >Assignee: Shinichiro Abe >Priority: Major > > Hello, > [Link to Apache mail > archive|http://mail-archives.apache.org/mod_mbox/manifoldcf-user/201804.mbox/%3C079e01d3d7da%24807b8f60%248172ae20%24%40citya.com%3E] > When we used Argument option in ManifoldCF for SolrCloud, ManifoldCF add they > arguments on the POST request and not on the url parameters. So, for add a > (pre)processor or a post-processor with the url, it's not possible. > [SolrConfig > updateRequestProcessorChain|https://lucene.apache.org/solr/guide/6_6/config-api.html#ConfigAPI-Whatabout_updateRequestProcessorChain_] > [call > UpdateRequestProcessors|https://lucene.apache.org/solr/guide/6_6/update-request-processors.html#UpdateRequestProcessors-Processor_Post-ProcessorRequestParameters] > [Conf image|https://image.ibb.co/cZC8bn/jira_update_processor.png] > Solr response: > org.apache.solr.common.SolrException: ERROR: > [doc=file:/srvics01/ways_holding/gestion_ged/gerance/3573/201102081135_ENVOIDEVISPP.doc] > unknown field 'processor' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
