[jira] [Commented] (CONNECTORS-1740) Solr 9 output connector
[ https://issues.apache.org/jira/browse/CONNECTORS-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773278#comment-17773278 ] Julien Massiera commented on CONNECTORS-1740: - Bonjour, Merci pour votre email. Mr Massiera a quitte ses fonctions le 14 avril 2023 et cet email sera bientot desactive. Pour toute question sur France Labs ou sur des projets en cours, merci de contacter cedric.ulmer att francelabs.com > Solr 9 output connector > --- > > Key: CONNECTORS-1740 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1740 > Project: ManifoldCF > Issue Type: Improvement > Components: Lucene/SOLR connector >Affects Versions: ManifoldCF 2.23 >Reporter: Julien Massiera >Assignee: Julien Massiera >Priority: Major > Attachments: CONNECTORS-1740.patch > > > The current Solr output connector is not compatible with Solr 9.x > We need to update the connector with SolrJ 9 and make sure that the custom > code (multipart post requests, basic/preemptive auth) is still required, and, > in case it is, port it ! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CONNECTORS-1740) Solr 9 output connector
[ https://issues.apache.org/jira/browse/CONNECTORS-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773277#comment-17773277 ] Guylaine Bassette commented on CONNECTORS-1740: --- Hello [~kwri...@metacarta.com], Thanks to have merged my Zookeeper fix ! We have tested it in our application and everything works well: job configuration and Solr indexation. > Solr 9 output connector > --- > > Key: CONNECTORS-1740 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1740 > Project: ManifoldCF > Issue Type: Improvement > Components: Lucene/SOLR connector >Affects Versions: ManifoldCF 2.23 >Reporter: Julien Massiera >Assignee: Julien Massiera >Priority: Major > Attachments: CONNECTORS-1740.patch > > > The current Solr output connector is not compatible with Solr 9.x > We need to update the connector with SolrJ 9 and make sure that the custom > code (multipart post requests, basic/preemptive auth) is still required, and, > in case it is, port it ! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CONNECTORS-1740) Solr 9 output connector
[ https://issues.apache.org/jira/browse/CONNECTORS-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773268#comment-17773268 ] Julien Massiera commented on CONNECTORS-1740: - Bonjour, Merci pour votre email. Mr Massiera a quitte ses fonctions le 14 avril 2023 et cet email sera bientot desactive. Pour toute question sur France Labs ou sur des projets en cours, merci de contacter cedric.ulmer att francelabs.com > Solr 9 output connector > --- > > Key: CONNECTORS-1740 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1740 > Project: ManifoldCF > Issue Type: Improvement > Components: Lucene/SOLR connector >Affects Versions: ManifoldCF 2.23 >Reporter: Julien Massiera >Assignee: Julien Massiera >Priority: Major > Attachments: CONNECTORS-1740.patch > > > The current Solr output connector is not compatible with Solr 9.x > We need to update the connector with SolrJ 9 and make sure that the custom > code (multipart post requests, basic/preemptive auth) is still required, and, > in case it is, port it ! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CONNECTORS-1740) Solr 9 output connector
[ https://issues.apache.org/jira/browse/CONNECTORS-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773266#comment-17773266 ] Guylaine Bassette commented on CONNECTORS-1740: --- Hello [~mingchun.zhao], Thanks for your contribution with Java 11 ! (y) I have integrated your patch with my contribution to fix the Zookeeper bug (PR 150, al ready merged to the master Branch). > Solr 9 output connector > --- > > Key: CONNECTORS-1740 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1740 > Project: ManifoldCF > Issue Type: Improvement > Components: Lucene/SOLR connector >Affects Versions: ManifoldCF 2.23 >Reporter: Julien Massiera >Assignee: Julien Massiera >Priority: Major > Attachments: CONNECTORS-1740.patch > > > The current Solr output connector is not compatible with Solr 9.x > We need to update the connector with SolrJ 9 and make sure that the custom > code (multipart post requests, basic/preemptive auth) is still required, and, > in case it is, port it ! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CONNECTORS-1740) Solr 9 output connector
[ https://issues.apache.org/jira/browse/CONNECTORS-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744458#comment-17744458 ] Julien Massiera commented on CONNECTORS-1740: - Bonjour, Merci pour votre email. Mr Massiera a quitte ses fonctions le 14 avril 2023 et cet email sera bientot desactive. Pour toute question sur France Labs ou sur des projets en cours, merci de contacter cedric.ulmer att francelabs.com > Solr 9 output connector > --- > > Key: CONNECTORS-1740 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1740 > Project: ManifoldCF > Issue Type: Improvement > Components: Lucene/SOLR connector >Affects Versions: ManifoldCF 2.23 >Reporter: Julien Massiera >Assignee: Julien Massiera >Priority: Major > Attachments: CONNECTORS-1740.patch > > > The current Solr output connector is not compatible with Solr 9.x > We need to update the connector with SolrJ 9 and make sure that the custom > code (multipart post requests, basic/preemptive auth) is still required, and, > in case it is, port it ! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CONNECTORS-1740) Solr 9 output connector
[ https://issues.apache.org/jira/browse/CONNECTORS-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744455#comment-17744455 ] Guylaine Bassette commented on CONNECTORS-1740: --- Hello, We have created an issue to the ZooKeeper Team : https://issues.apache.org/jira/browse/ZOOKEEPER-4722. We provided them a ManifoldCF fork project they can build to reproduce the bug (https://gitlab.datafari.com/sandboxespublic/manifoldcf). > Solr 9 output connector > --- > > Key: CONNECTORS-1740 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1740 > Project: ManifoldCF > Issue Type: Improvement > Components: Lucene/SOLR connector >Affects Versions: ManifoldCF 2.23 >Reporter: Julien Massiera >Assignee: Julien Massiera >Priority: Major > Attachments: CONNECTORS-1740.patch > > > The current Solr output connector is not compatible with Solr 9.x > We need to update the connector with SolrJ 9 and make sure that the custom > code (multipart post requests, basic/preemptive auth) is still required, and, > in case it is, port it ! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CONNECTORS-1740) Solr 9 output connector
[ https://issues.apache.org/jira/browse/CONNECTORS-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729127#comment-17729127 ] Julien Massiera commented on CONNECTORS-1740: - Bonjour, Merci pour votre email. Mr Massiera a quitte ses fonctions le 14 avril 2023 et cet email sera bientot desactive. Pour toute question sur France Labs ou sur des projets en cours, merci de contacter cedric.ulmer att francelabs.com > Solr 9 output connector > --- > > Key: CONNECTORS-1740 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1740 > Project: ManifoldCF > Issue Type: Improvement > Components: Lucene/SOLR connector >Affects Versions: ManifoldCF 2.23 >Reporter: Julien Massiera >Assignee: Julien Massiera >Priority: Major > Attachments: CONNECTORS-1740.patch > > > The current Solr output connector is not compatible with Solr 9.x > We need to update the connector with SolrJ 9 and make sure that the custom > code (multipart post requests, basic/preemptive auth) is still required, and, > in case it is, port it ! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CONNECTORS-1740) Solr 9 output connector
[ https://issues.apache.org/jira/browse/CONNECTORS-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729126#comment-17729126 ] Mingchun Zhao commented on CONNECTORS-1740: --- Hi [~kwri...@metacarta.com] [~julienFL] , I’ve tried to modified the source code for using Zookeeper `3.5.6` and migrating Java from 8 to `11`, and the test run-tests-framework was OK. Attached the patch [^CONNECTORS-1740.patch] I used in my test. Also, I tried various other Zookeeper versions(3.6.2, 3.7.0, 3.7.2, 3.8.0, 3.8.1), but all of them failed in the test run-tests-framework due to an error as below. ``` [junit] [reader] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=localhost:8348 sessionTimeout=2000 watcher=org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection$ZooKeeperWatcher@b647be3 [junit] [reader] INFO org.apache.zookeeper.ClientCnxnSocket - jute.maxbuffer value is 1048575 Bytes [junit] [reader] INFO org.apache.zookeeper.ClientCnxn - zookeeper.request.timeout value is 0. feature enabled=false [junit] [reader-SendThread(localhost:8348)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/0:0:0:0:0:0:0:1:8348. [junit] [reader-SendThread(localhost:8348)] INFO org.apache.zookeeper.ClientCnxn - SASL config status: Will not attempt to authenticate using SASL (unknown error) [junit] [reader-SendThread(localhost:8348)] WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for sever localhost/0:0:0:0:0:0:0:1:8348, Closing socket connection. Attempting reconnect except it is a SessionExpiredException. [junit] java.net.ConnectException: Connection refused [junit] at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [junit] at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777) [junit] at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:344) [junit] at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1280) ``` > Solr 9 output connector > --- > > Key: CONNECTORS-1740 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1740 > Project: ManifoldCF > Issue Type: Improvement > Components: Lucene/SOLR connector >Affects Versions: ManifoldCF 2.23 >Reporter: Julien Massiera >Assignee: Julien Massiera >Priority: Major > Attachments: CONNECTORS-1740.patch > > > The current Solr output connector is not compatible with Solr 9.x > We need to update the connector with SolrJ 9 and make sure that the custom > code (multipart post requests, basic/preemptive auth) is still required, and, > in case it is, port it ! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CONNECTORS-1740) Solr 9 output connector
[ https://issues.apache.org/jira/browse/CONNECTORS-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729125#comment-17729125 ] Julien Massiera commented on CONNECTORS-1740: - Bonjour, Merci pour votre email. Mr Massiera a quitte ses fonctions le 14 avril 2023 et cet email sera bientot desactive. Pour toute question sur France Labs ou sur des projets en cours, merci de contacter cedric.ulmer att francelabs.com > Solr 9 output connector > --- > > Key: CONNECTORS-1740 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1740 > Project: ManifoldCF > Issue Type: Improvement > Components: Lucene/SOLR connector >Affects Versions: ManifoldCF 2.23 >Reporter: Julien Massiera >Assignee: Julien Massiera >Priority: Major > Attachments: CONNECTORS-1740.patch > > > The current Solr output connector is not compatible with Solr 9.x > We need to update the connector with SolrJ 9 and make sure that the custom > code (multipart post requests, basic/preemptive auth) is still required, and, > in case it is, port it ! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CONNECTORS-1740) Solr 9 output connector
[ https://issues.apache.org/jira/browse/CONNECTORS-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17712355#comment-17712355 ] Julien Massiera commented on CONNECTORS-1740: - No I did not try with an older Zookeeper version, I used the recommended one specified for Solr 9. We can indeed test this ! > Solr 9 output connector > --- > > Key: CONNECTORS-1740 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1740 > Project: ManifoldCF > Issue Type: Improvement > Components: Lucene/SOLR connector >Affects Versions: ManifoldCF 2.23 >Reporter: Julien Massiera >Assignee: Julien Massiera >Priority: Major > > The current Solr output connector is not compatible with Solr 9.x > We need to update the connector with SolrJ 9 and make sure that the custom > code (multipart post requests, basic/preemptive auth) is still required, and, > in case it is, port it ! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CONNECTORS-1740) Solr 9 output connector
[ https://issues.apache.org/jira/browse/CONNECTORS-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17712329#comment-17712329 ] Karl Wright commented on CONNECTORS-1740: - It looks like there's an existing and unresolved Zookeeper issue for this: https://issues.apache.org/jira/browse/ZOOKEEPER-3828 Unfortunately this makes going to Solr 9 a no-go, unless we can we use an older version of Zookeeper without this problem. Have you tried that? > Solr 9 output connector > --- > > Key: CONNECTORS-1740 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1740 > Project: ManifoldCF > Issue Type: Improvement > Components: Lucene/SOLR connector >Affects Versions: ManifoldCF 2.23 >Reporter: Julien Massiera >Assignee: Julien Massiera >Priority: Major > > The current Solr output connector is not compatible with Solr 9.x > We need to update the connector with SolrJ 9 and make sure that the custom > code (multipart post requests, basic/preemptive auth) is still required, and, > in case it is, port it ! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CONNECTORS-1740) Solr 9 output connector
[
https://issues.apache.org/jira/browse/CONNECTORS-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17712327#comment-17712327
]
Karl Wright commented on CONNECTORS-1740:
-
Zookeeper tests are failing because Zookeeper is not doing what it is supposed
to. Basically, sessions are being destroyed so fast that the test cannot
actually do anything useful:
{code}
[junit] org.apache.zookeeper.ClientCnxn$SessionTimeoutException: Client
session timed out, have not heard from server in 2007ms for session id 0x0
[junit] at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1250)
[junit] [reader] INFO org.apache.zookeeper.ZooKeeper - Session: 0x0 closed
[junit] [reader-EventThread] INFO org.apache.zookeeper.ClientCnxn -
EventThread shut down for session: 0x0
[junit] [reader] INFO org.apache.zookeeper.ZooKeeper - Initiating client
connection, connectString=localhost:8348 sessionTimeout=2000
watcher=org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection$ZooKeeperWatcher@45c66a27
[junit] [reader] INFO org.apache.zookeeper.ClientCnxnSocket -
jute.maxbuffer value is 1048575 Bytes
[junit] [reader] INFO org.apache.zookeeper.ClientCnxn -
zookeeper.request.timeout value is 0. feature enabled=false
[junit] [reader-SendThread(localhost:8348)] INFO
org.apache.zookeeper.ClientCnxn - Opening socket connection to server
localhost/127.0.0.1:8348.
[junit] [reader-SendThread(localhost:8348)] INFO
org.apache.zookeeper.ClientCnxn - SASL config status: Will not attempt to
authenticate using SASL (unknown error)
[junit] [reader-SendThread(localhost:8348)] WARN
org.apache.zookeeper.ClientCnxn - Client session timed out, have not heard from
server in 2008ms for session id 0x0
[junit] [reader-SendThread(localhost:8348)] WARN
org.apache.zookeeper.ClientCnxn - An exception was thrown while closing send
thread for session 0x0.
[junit] org.apache.zookeeper.ClientCnxn$SessionTimeoutException: Client
session timed out, have not heard from server in 2008ms for session id 0x0
{code}
This is, of course, fatal to ManifoldCF's use of Zookeeper for synchronization.
So we need to figure out how to work around this new "feature", or the whole
project is a no-go.
> Solr 9 output connector
> ---
>
> Key: CONNECTORS-1740
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1740
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Lucene/SOLR connector
>Affects Versions: ManifoldCF 2.23
>Reporter: Julien Massiera
>Assignee: Julien Massiera
>Priority: Major
>
> The current Solr output connector is not compatible with Solr 9.x
> We need to update the connector with SolrJ 9 and make sure that the custom
> code (multipart post requests, basic/preemptive auth) is still required, and,
> in case it is, port it !
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CONNECTORS-1740) Solr 9 output connector
[
https://issues.apache.org/jira/browse/CONNECTORS-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17712321#comment-17712321
]
Karl Wright commented on CONNECTORS-1740:
-
Turns out that this update also will require we move permanently from Java 8 to
Java 11:
{code}
compile-connector:
[javac] Compiling 17 source files to
C:\wip\mcf\CONNECTORS-1740\connectors\solr\build\connector\classes
[javac]
C:\wip\mcf\CONNECTORS-1740\connectors\solr\connector\src\main\java\org\apache\manifoldcf\agents\output\solr\ModifiedHttp2SolrClient.java:3:
error: cannot access Utils
[javac] import static org.apache.solr.common.util.Utils.getObjectByPath;
[javac] ^
[javac] bad class file:
C:\wip\mcf\CONNECTORS-1740\lib\solr-solrj-9.1.0.jar(org/apache/solr/common/util/Utils.class)
[javac] class file has wrong version 55.0, should be 52.0
[javac] Please remove or make sure it appears in the correct
subdirectory of the classpath.
{code}
> Solr 9 output connector
> ---
>
> Key: CONNECTORS-1740
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1740
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Lucene/SOLR connector
>Affects Versions: ManifoldCF 2.23
>Reporter: Julien Massiera
>Assignee: Julien Massiera
>Priority: Major
>
> The current Solr output connector is not compatible with Solr 9.x
> We need to update the connector with SolrJ 9 and make sure that the custom
> code (multipart post requests, basic/preemptive auth) is still required, and,
> in case it is, port it !
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CONNECTORS-1740) Solr 9 output connector
[ https://issues.apache.org/jira/browse/CONNECTORS-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711593#comment-17711593 ] Karl Wright commented on CONNECTORS-1740: - [~julienFL], I'll look at the zookeeper changes. > Solr 9 output connector > --- > > Key: CONNECTORS-1740 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1740 > Project: ManifoldCF > Issue Type: Improvement > Components: Lucene/SOLR connector >Affects Versions: ManifoldCF 2.23 >Reporter: Julien Massiera >Assignee: Julien Massiera >Priority: Major > > The current Solr output connector is not compatible with Solr 9.x > We need to update the connector with SolrJ 9 and make sure that the custom > code (multipart post requests, basic/preemptive auth) is still required, and, > in case it is, port it ! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CONNECTORS-1740) Solr 9 output connector
[ https://issues.apache.org/jira/browse/CONNECTORS-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711425#comment-17711425 ] Julien Massiera commented on CONNECTORS-1740: - r1909097 of the branch, the Solr output connector has been fixed and now multipart is applied on all POST requests The Unit tests of framework core (Zookeeper tests) and Solr connector are still broken. I would really appreciate help to fix the tests ! > Solr 9 output connector > --- > > Key: CONNECTORS-1740 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1740 > Project: ManifoldCF > Issue Type: Improvement > Components: Lucene/SOLR connector >Affects Versions: ManifoldCF 2.23 >Reporter: Julien Massiera >Priority: Major > > The current Solr output connector is not compatible with Solr 9.x > We need to update the connector with SolrJ 9 and make sure that the custom > code (multipart post requests, basic/preemptive auth) is still required, and, > in case it is, port it ! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CONNECTORS-1740) Solr 9 output connector
[ https://issues.apache.org/jira/browse/CONNECTORS-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643980#comment-17643980 ] Julien Massiera commented on CONNECTORS-1740: - I created a CONNECTORS-1740 branch. I updated the SolrJ version to 9 and Zookeeper to 3.8.0 Some problems appear: * The tests of the framework core are broken because of Zookeeper * The tests of the Solr connector are broken * The updated Solr output connector works with Solr 9 and older versions but I did not port the basic/preemptive auth, neither the multipart post requests custom code. After some tests it appears that they are required because some documents with a lot of metadata trigger errors during the ingest phase Unfortunately I currently don't have more time to spend on these issues and I would appreciate any help to solve them ! > Solr 9 output connector > --- > > Key: CONNECTORS-1740 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1740 > Project: ManifoldCF > Issue Type: Improvement > Components: Lucene/SOLR connector >Affects Versions: ManifoldCF 2.23 >Reporter: Julien Massiera >Priority: Major > > The current Solr output connector is not compatible with Solr 9.x > We need to update the connector with SolrJ 9 and make sure that the custom > code (multipart post requests, basic/preemptive auth) is still required, and, > in case it is, port it ! -- This message was sent by Atlassian Jira (v8.20.10#820010)
