I see you have committed this. You missed a few spots; the extra jar was
mentioned in multiple places. I committed another fix to correct that.
Karl
On Wed, Oct 25, 2023 at 10:46 AM Guylaine BASSETTE <
guylaine.basse...@francelabs.com> wrote:
> Hello Karl,
>
> Thank you very much for this update! I have tested your suggestions and
> kafka ITs tests ended successfully. :-)
>
> Here you can find the patch.
>
> My only doubt is this warning I had in some parts of the kafka test:
>
> ```
>
> [junit] [Controller-0-to-broker-0-send-thread] INFO
> org.apache.kafka.clients.NetworkClient - [Controller id=0,
> targetBrokerId=0] Node 0 disconnected.
> [junit] [Controller-0-to-broker-0-send-thread] WARN
> org.apache.kafka.clients.NetworkClient - [Controller id=0,
> targetBrokerId=0] Connection to node 0 (guylaine-virtual-machine/
> 127.0.1.1:9092) could not be established. Broker may not be available.
> [junit] [Controller-0-to-broker-0-send-thread] WARN
> kafka.controller.RequestSendThread - [RequestSendThread controllerId=0]
> Controller 0's connection to broker guylaine-virtual-machine:9092 (id: 0
> rack: null) was unsuccessful
> [junit] java.io.IOException: Connection to
> guylaine-virtual-machine:9092 (id: 0 rack: null) failed.
> [junit] at
> org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
> [junit] at
> kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:298)
> [junit] at
> kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:251)
> [junit] at
> org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:130)
> [junit] [Controller-0-to-broker-0-send-thread] INFO
> org.apache.kafka.clients.NetworkClient - [Controller id=0,
> targetBrokerId=0] Client requested connection close from node 0
> [junit] [controller-event-thread] INFO state.change.logger -
> [Controller id=0 epoch=2] Sending LeaderAndIsr request to broker 0 with 1
> become-leader and 0 become-follower partitions
> ```
>
> I have also ran an "ant test". It run core-framework and ITs tests until
> mongoDB connector with this fail:
> ```
> [junit] Testcase:
> sanityCheck(org.apache.manifoldcf.agents.output.mongodboutput.tests.APISanityHSQLDBIT):
> Caused an ERROR
> [junit] Could not start process: <EOF>
> [junit] java.lang.RuntimeException: Could not start process: <EOF>
> [junit] at
> de.flapdoodle.embed.mongo.AbstractMongoProcess.onAfterProcessStart(AbstractMongoProcess.java:81)
> [junit] at
> de.flapdoodle.embed.process.runtime.AbstractProcess.<init>(AbstractProcess.java:115)
> [junit] at
> de.flapdoodle.embed.mongo.AbstractMongoProcess.<init>(AbstractMongoProcess.java:54)
> [junit] at
> de.flapdoodle.embed.mongo.MongodProcess.<init>(MongodProcess.java:50)
> [junit] at
> de.flapdoodle.embed.mongo.MongodExecutable.start(MongodExecutable.java:44)
> [junit] at
> de.flapdoodle.embed.mongo.MongodExecutable.start(MongodExecutable.java:34)
> [junit] at
> de.flapdoodle.embed.process.runtime.Executable.start(Executable.java:109)
> [junit] at
> org.apache.manifoldcf.agents.output.mongodboutput.tests.BaseITHSQLDB.setUpMongoDB(BaseITHSQLDB.java:72)
> [junit] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> [junit] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit]
> [junit]
> [junit] Testcase:
> sanityCheck(org.apache.manifoldcf.agents.output.mongodboutput.tests.APISanityHSQLDBIT):
> Caused an ERROR
> [junit] null
> [junit] java.lang.NullPointerException
> [junit] at
> org.apache.manifoldcf.agents.output.mongodboutput.tests.APISanityHSQLDBIT.removeTestArea(APISanityHSQLDBIT.java:109)
> [junit] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> [junit] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit] at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> ```
>
> And no more success for the ITs tests of Solr connector...
>
> Guylaine
> Le 25/10/2023 à 12:52, Karl Wright a écrit :
>
> I was able to reproduce the problem last night. I believe the cause may
> well be that we've moved too many dependencies to the framework level.
> Specifically, I think perhaps only zookeeper and netty should run there,
> but the Scala library probably needs to run at the same classloader level
> as the other Scala jars, so it should be moved back into connectors/kafka
> and removed from framework/build.xml and from build.xml and from
> connector-build.xml.
>
> Sadly I'm completely snowed under until the weekend so it will need to wait
> until then unless someone else wants to try this.
>
> Karl
>
>
> On Tue, Oct 24, 2023 at 9:12 AM Karl Wright <daddy...@gmail.com>
> <daddy...@gmail.com> wrote:
>
>
> Try doing svn update and deleting your test-materials directory contents.
> Then ant download-dependencies. You don't get a link error after that when
> you do run-IT-HSQLDB .
>
> Karl
>
> On Tue, Oct 24, 2023 at 9:10 AM Guylaine BASSETTE
> <guylaine.basse...@francelabs.com> wrote:
>
>
> Hi Karl and Mingchun,
>
> Thanks again !
>
> My bad for the Zookeeper dependencies. Actually I've made a mistake
> using in my IDE dependencies analyzer.
>
> Regarding Kafka tests, a light search brings me to the
> "spark-streaming-kafka" dependency that might be missing...
>
> At the mean time, I continue my effort on Solr connector tests.
>
>
> For any use, here is the errors I get:
>
> ```
>
> [junit] [main] INFO org.eclipse.jetty.server.handler.ContextHandler -
> Stopped o.e.j.w.WebAppContext@7c28c1{ManifoldCF General API
>
> Webapp,/mcf-api-service,null,STOPPED}{/home/guylaine/IdeaProjects/mon-manifoldcf/dist/web/war/mcf-api-service.war}
> [junit] [main] INFO org.eclipse.jetty.server.handler.ContextHandler
> - Stopped o.e.j.w.WebAppContext@588ffeb{ManifoldCF Authorities API
>
> Webapp,/mcf-authority-service,null,STOPPED}{/home/guylaine/IdeaProjects/mon-manifoldcf/dist/web/war/mcf-authority-service.war}
> [junit] [main] INFO org.eclipse.jetty.server.handler.ContextHandler
> - Stopped o.e.j.w.WebAppContext@71a3a190{ManifoldCF Crawler
>
> Interface,/mcf-crawler-ui,null,STOPPED}{/home/guylaine/IdeaProjects/mon-manifoldcf/dist/web/war/mcf-crawler-ui.war}
> [junit] ------------- ---------------- ---------------
> [junit] Testcase:
> sanityCheck(org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT):
> Caused an ERROR
> [junit] 'scala.collection.immutable.ArraySeq
> scala.runtime.ScalaRunTime$.wrapRefArray(java.lang.Object[])'
> [junit] java.lang.NoSuchMethodError:
> 'scala.collection.immutable.ArraySeq
> scala.runtime.ScalaRunTime$.wrapRefArray(java.lang.Object[])'
> [junit] at
> kafka.server.KafkaConfig$.<clinit>(KafkaConfig.scala:338)
> [junit] at
> kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1603)
> [junit] at
>
> org.apache.manifoldcf.agents.output.kafka.KafkaLocal.<init>(KafkaLocal.java:31)
> [junit] at
>
> org.apache.manifoldcf.agents.output.kafka.BaseITHSQLDB.setupKafka(BaseITHSQLDB.java:86)
> [junit] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> [junit] at
>
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit] at
>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit]
> [junit]
> [junit] Testcase:
> sanityCheck(org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT):
> Caused an ERROR
> [junit] null
> [junit] java.lang.NullPointerException
> [junit] at
>
> org.apache.manifoldcf.agents.output.kafka.BaseITHSQLDB.cleanUpKafka(BaseITHSQLDB.java:92)
> [junit] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> [junit] at
>
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [junit] at
>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> ```
>
> Le 24/10/2023 à 12:30, Karl Wright a écrit :
>
> I missed a place - connector-build.xml. Updated now.
>
> Now we don't get a link exception, but neither does the kafka test work.
> It seems to be unable to start zookeeper even though all the
>
> dependencies
>
> are now there. Will need to look at this after work.
>
> Karl
>
>
> On Mon, Oct 23, 2023 at 11:32 PM Mingchun Zhao<
>
> mingchun.zha...@gmail.com>
>
> wrote:
>
>
> Thanks. I've tried `ant test` with the latest trunk. As a result, the
>
> kafka
>
> test failed as below.
> ```
> [junit] Testcase:
>
>
> sanityCheck(org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT):
>
> Caused an ERROR
> [junit] io/netty/handler/ssl/SslContext
> [junit] java.lang.NoClassDefFoundError:
>
> io/netty/handler/ssl/SslContext
>
> [junit] at
>
>
>
> org.apache.zookeeper.common.ZKConfig.handleBackwardCompatibility(ZKConfig.java:106)
>
> [junit] at
>
>
>
> org.apache.zookeeper.client.ZKClientConfig.handleBackwardCompatibility(ZKClientConfig.java:96)
>
> [junit] at
> org.apache.zookeeper.common.ZKConfig.init(ZKConfig.java:92)
> [junit] at
> org.apache.zookeeper.common.ZKConfig.<init>(ZKConfig.java:61)
> [junit] at
>
>
> org.apache.zookeeper.client.ZKClientConfig.<init>(ZKClientConfig.java:69)
>
> [junit] at
>
> kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1620)
>
> [junit] at
>
> kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1603)
>
> [junit] at
>
>
>
> org.apache.manifoldcf.agents.output.kafka.KafkaLocal.<init>(KafkaLocal.java:31)
>
> [junit] at
>
>
>
> org.apache.manifoldcf.agents.output.kafka.BaseITHSQLDB.setupKafka(BaseITHSQLDB.java:86)
>
> [junit] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> [junit] at
>
>
>
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>
> [junit] at
>
>
>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> [junit] Caused by: java.lang.ClassNotFoundException:
> io.netty.handler.ssl.SslContext
> [junit] at
>
>
>
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
>
> [junit] at
>
>
>
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>
> [junit] at
> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
> [junit]
> [junit]
> [junit] Testcase:
>
>
> sanityCheck(org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT):
>
> Caused an ERROR
> [junit] null
> [junit] java.lang.NullPointerException
> [junit] at
>
>
>
> org.apache.manifoldcf.agents.output.kafka.BaseITHSQLDB.cleanUpKafka(BaseITHSQLDB.java:92)
>
> [junit] at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> [junit] at
>
>
>
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>
> [junit] at
>
>
>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> [junit]
> [junit]
>
> BUILD FAILED
> /Users/zhaomingchun/ManifoldCF/manifoldcf/build.xml:517: The following
> error occurred while executing this line:
> /Users/zhaomingchun/ManifoldCF/manifoldcf/build.xml:471: The following
> error occurred while executing this line:
>
>
> /Users/zhaomingchun/ManifoldCF/manifoldcf/dist/connector-build.xml:1102:
>
> Test org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT failed
> ```
>
> 2023年10月24日(火) 11:00 Karl Wright<daddy...@gmail.com> <daddy...@gmail.com>:
>
>
> Okay, I updated zookeeper properly in build.xml and
>
> framework/build.xml,
>
> with the two new dependencies, and the zookeeper tests pass. I
>
> haven't
>
> tried the kafka or solr tests yet.
>
> Karl
>
>
> On Mon, Oct 23, 2023 at 9:29 PM Karl Wright<daddy...@gmail.com>
> <daddy...@gmail.com>
>
> wrote:
>
> Unless I know what kafka is using zookeeper for, this would seem
>
> risky
>
> to
>
> me. Zookeeper is meant to coordinate processes; it may not work for
>
> one
>
> process to be using different versions of zookeeper than the others.
>
> It looks like the original change to kafka you reverted had the
>
> proper
>
> dependencies but they absolutely needed to be included in the right
> classpaths and they weren't - they were only included in the kafka
>
> tests.
>
> I will look at this perhaps at the latest this weekend, but I won't
>
> commit
>
> this patch.
>
>
> Karl
>
>
> On Mon, Oct 23, 2023 at 5:14 PM Mingchun Zhao <
>
> mingchun.zha...@gmail.com
>
> wrote:
>
>
> I reverted zookeeper version to 3.8.0 to avoid linkage error on the
> multiThreadZooKeeperLockTest:
> [junit] Caused by: java.lang.ClassNotFoundException:
> io.netty.handler.ssl.SslContext
>
> I've prepared a PR here:https://github.com/apache/manifoldcf/pull/156
>
> Just a heads up, `ant test` still hangs on the Solr Output connector
>
> test:
>
> ```
> run-IT-HSQLDB:
> [junit] Testsuite:
> org.apache.manifoldcf.agents.output.solr.tests.SolrCrawlHSQLDBIT
> [junit] Configuration file successfully read
> [junit] [main] INFO org.eclipse.jetty.util.log - Logging
>
> initialized
>
> @7027ms to org.eclipse.jetty.util.log.Slf4jLog
> [junit] [main] INFO org.eclipse.jetty.server.Server -
> jetty-9.4.48.v20220622; built: 2022-06-21T20:42:25.880Z; git:
> 6b67c5719d1f4371b33655ff2d047d24e171e49a; jvm 11.0.11+9
> [junit] [main] INFO org.eclipse.jetty.server.session -
> DefaultSessionIdManager workerName=node0
> [junit] [main] INFO org.eclipse.jetty.server.session - No
> SessionScavenger set, using defaults
> [junit] [main] INFO org.eclipse.jetty.server.session - node0
> Scavenging
> every 660000ms
> [junit] [main] INFO
>
> org.eclipse.jetty.server.handler.ContextHandler
>
> -
>
> Started o.e.j.w.WebAppContext@1517f633{ManifoldCF Crawler
>
>
>
> Interface,/mcf-crawler-ui,file:///private/var/folders/zh/mx4q_qh93cv6jtp13ht8b1fr0000gn/T/jetty-0_0_0_0-8346-mcf-crawler-ui_war-_mcf-crawler-ui-any-7554899724821045499/webapp/,AVAILABLE}{/Users/zhaomingchun/ManifoldCF/manifoldcf/dist/web/war/mcf-crawler-ui.war}
>
> [junit] [main] INFO
>
> org.eclipse.jetty.server.handler.ContextHandler
>
> -
>
> Started o.e.j.w.WebAppContext@4fe01803{ManifoldCF Authorities API
>
>
>
> Webapp,/mcf-authority-service,file:///private/var/folders/zh/mx4q_qh93cv6jtp13ht8b1fr0000gn/T/jetty-0_0_0_0-8346-mcf-authority-service_war-_mcf-authority-service-any-7701836901953162228/webapp/,AVAILABLE}{/Users/zhaomingchun/ManifoldCF/manifoldcf/dist/web/war/mcf-authority-service.war}
>
> [junit] Creating mock service
> [junit] Mock service created
> [junit] [main] INFO
>
> org.eclipse.jetty.server.handler.ContextHandler
>
> -
>
> Started o.e.j.w.WebAppContext@13d186db{ManifoldCF General API
>
>
>
> Webapp,/mcf-api-service,file:///private/var/folders/zh/mx4q_qh93cv6jtp13ht8b1fr0000gn/T/jetty-0_0_0_0-8346-mcf-api-service_war-_mcf-api-service-any-2609388202403972652/webapp/,AVAILABLE}{/Users/zhaomingchun/ManifoldCF/manifoldcf/dist/web/war/mcf-api-service.war}
>
> [junit] [main] INFO org.eclipse.jetty.server.AbstractConnector
>
> -
>
> Started ServerConnector@3bd55d8{HTTP/1.1, (http/1.1)}{0.0.0.0:8346}
> [junit] [main] INFO org.eclipse.jetty.server.Server - Started
>
> @9054ms
>
> [junit] [main] INFO org.eclipse.jetty.server.Server -
> jetty-9.4.48.v20220622; built: 2022-06-21T20:42:25.880Z; git:
> 6b67c5719d1f4371b33655ff2d047d24e171e49a; jvm 11.0.11+9
> [junit] [main] INFO org.eclipse.jetty.server.session -
> DefaultSessionIdManager workerName=node0
> [junit] [main] INFO org.eclipse.jetty.server.session - No
> SessionScavenger set, using defaults
> [junit] [main] INFO org.eclipse.jetty.server.session - node0
> Scavenging
> every 600000ms
> [junit] [main] INFO
>
> org.eclipse.jetty.server.handler.ContextHandler
>
> -
>
> Started o.e.j.s.ServletContextHandler@6f4ade6e
>
> {/solr,null,AVAILABLE}
>
> [junit] [main] INFO org.eclipse.jetty.server.AbstractConnector
>
> -
>
> Started ServerConnector@30e6a763{HTTP/1.1, (http/1.1)}{0.0.0.0:8188
>
> }
>
> [junit] [main] INFO org.eclipse.jetty.server.Server - Started
>
> @9064ms
>
> [junit] [main] INFO org.eclipse.jetty.server.AbstractConnector
>
> -
>
> Stopped ServerConnector@30e6a763{HTTP/1.1, (http/1.1)}{0.0.0.0:8188
>
> }
>
> [junit] [main] INFO org.eclipse.jetty.server.session - node0
>
> Stopped
>
> scavenging
> [junit] [main] INFO
>
> org.eclipse.jetty.server.handler.ContextHandler
>
> -
>
> Stopped o.e.j.s.ServletContextHandler@6f4ade6e{/solr,null,STOPPED}
> ```
>
> 2023年10月24日(火) 1:40 Karl Wright<daddy...@gmail.com> <daddy...@gmail.com>:
>
>
> The dependencies would be in the zookeeper pom. Maven would follow
>
> them
>
> automatically which is why it is insufficient to assume that if
>
> maven
>
> works
>
> so will ant.
>
> You can use mvn dependency:tree to find what Maven is actually
>
> pulling
>
> in.
>
> Karl
>
>
> On Mon, Oct 23, 2023 at 11:04 AM Guylaine BASSETTE
> <guylaine.basse...@francelabs.com> wrote:
>
>
> I launch these tests with a Maven build and everything is OK. Yet
>
> no
>
> netty dependencies are requiered... I don't understand were this
> SslContext is called...
>
> Le 23/10/2023 à 16:25, Karl Wright a écrit :
>
> Yes, that is indicating that zookeeper is looking for a specific
>
> netty
>
> class that it isn't finding. That is why I think there is now a
>
> zookeeper
>
> dependency we aren't including in the classpaths that include
>
> zookeeper.
>
> Karl
>
>
> On Mon, Oct 23, 2023 at 10:23 AM Mingchun Zhao<
>
> mingchun.zha...@gmail.com
>
> wrote:
>
>
> Karl, Thanks!
> I think I reproduced that multiThreadZooKeeperLockTest error
>
> when I
>
> running `ant test`, will look into this.
> ```
> [junit] ------------- ---------------- ---------------
> [junit] Testcase:
>
>
>
> multiThreadZooKeeperLockTest(org.apache.manifoldcf.core.lockmanager.TestZooKeeperLocks):
>
> Caused an ERROR
> [junit] io/netty/handler/ssl/SslContext
> [junit] java.lang.NoClassDefFoundError:
>
> io/netty/handler/ssl/SslContext
>
> [junit] at
>
>
>
> org.apache.zookeeper.common.ZKConfig.handleBackwardCompatibility(ZKConfig.java:106)
>
> [junit] at
>
>
>
> org.apache.zookeeper.client.ZKClientConfig.handleBackwardCompatibility(ZKClientConfig.java:96)
>
> [junit] at
> org.apache.zookeeper.common.ZKConfig.init(ZKConfig.java:92)
> [junit] at
> org.apache.zookeeper.common.ZKConfig.<init>(ZKConfig.java:61)
> [junit] at
>
>
> org.apache.zookeeper.client.ZKClientConfig.<init>(ZKClientConfig.java:69)
>
> [junit] at
> org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:643)
> [junit] at
> org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:567)
> [junit] at
> org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:734)
> [junit] at
> org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:448)
> [junit] at
>
>
>
> org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.createSession(ZooKeeperConnection.java:74)
>
> [junit] at
>
>
>
> org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.<init>(ZooKeeperConnection.java:66)
>
> [junit] at
>
>
>
> org.apache.manifoldcf.core.lockmanager.ZooKeeperConnectionPool.grab(ZooKeeperConnectionPool.java:48)
>
> [junit] at
>
>
>
> org.apache.manifoldcf.core.lockmanager.ZooKeeperLockObject.obtainGlobalReadLock(ZooKeeperLockObject.java:190)
>
> [junit] at
>
>
>
> org.apache.manifoldcf.core.lockmanager.LockObject.enterReadLock(LockObject.java:310)
>
> [junit] at
>
>
>
> org.apache.manifoldcf.core.lockmanager.LockGate.enterReadLock(LockGate.java:271)
>
> [junit] at
>
>
>
> org.apache.manifoldcf.core.lockmanager.TestZooKeeperLocks.enterReadLock(TestZooKeeperLocks.java:125)
>
> [junit] at
>
>
>
> org.apache.manifoldcf.core.lockmanager.TestZooKeeperLocks$ReaderThread.run(TestZooKeeperLocks.java:204)
>
> [junit] Caused by: java.lang.ClassNotFoundException:
> io.netty.handler.ssl.SslContext
> [junit] at
>
>
>
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
>
> [junit] at
>
>
>
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>
> [junit] at
> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
> [junit]
> [junit]
>
> BUILD FAILED
> ```
>
> 2023年10月23日(月) 23:02 Karl Wright<daddy...@gmail.com> <daddy...@gmail.com>:
>
>
> This is all you need to do:
>
> ant clean-core-deps
> ant make-core-deps
> ant clean
> ant test
>
> Karl
>
>
> On Mon, Oct 23, 2023 at 9:55 AM Mingchun Zhao <
>
> mingchun.zha...@gmail.com
>
> wrote:
>
>
> Hi Guylaine, Thanks!
>
>
> Thanks for all your sharing, it's very helpful! I'll
>
> continue...
>
> I'll look into it some more too. If I have any other
>
> information
>
> I'll
>
> share
>
> it with you.
>
> 2023年10月23日(月) 22:49 Guylaine BASSETTE <
>
> guylaine.basse...@francelabs.com
>
> :
>
>
> Thanks for all your sharing, it's very helpful! I'll
>
> continue...
>
> Sorry, French and English mixed up!
>
> Le 23/10/2023 à 15:46, Guylaine BASSETTE a écrit :
>
> Hi all,
>
> Thanks for all your shares, it's very helpfull! Merci pour
>
> tous
>
> vos
>
> partages, c'est très utile ! Je poursuis...
>
> Le 23/10/2023 à 15:31, Karl Wright a écrit :
>
> I can't give advice on the test; this is something
>
> FranceLabs
>
> should
>
> look
> at.
> However, nothing of what you are doing will affect the
>
> zookeeper
>
> tests in
> framework. That's a totally different issue.
>
> Karl
>
>
> On Mon, Oct 23, 2023 at 9:20 AM Mingchun Zhao<
>
> mingchun.zha...@gmail.com
>
> wrote:
>
>
> Hi Karl and Guylaine,
>
>
> I hope and think it's just a problem specific to the
>
> test.
>
> Missing
>
> updates or incompatible dependencies...
>
> Allow me to share with you what I'm working on. I've
>
> tried
>
> to
>
> support
>
> http2C within the Solr output connector junit test, but
>
> got
>
> another
>
> unhandled solr exception when I ran `ant run-IT-HSQLDB`.
>
> - source code change
> ```
> diff --git
>
>
>
> a/connectors/solr/connector/src/test/java/org/apache/manifoldcf/agents/output/solr/tests
>
> /MockSolrService.java
>
>
>
> b/connectors/solr/connector/src/test/java/org/apache/manifoldcf/agents/output
>
> /solr/tests/MockSolrService.java
> index 237ade09c..3fb558f52 100644
> ---
>
>
>
> a/connectors/solr/connector/src/test/java/org/apache/manifoldcf/agents/output/solr/tests/MockSo
>
> lrService.java
> +++
>
>
>
> b/connectors/solr/connector/src/test/java/org/apache/manifoldcf/agents/output/solr/tests/MockSo
>
> lrService.java
> @@ -18,7 +18,10 @@
> */
> package
>
> org.apache.manifoldcf.agents.output.solr.tests;
>
> +import
>
> org.eclipse.jetty.http2.server.HTTP2CServerConnectionFactory;
>
> import org.eclipse.jetty.servlet.ServletHolder;
> +import org.eclipse.jetty.server.HttpConfiguration;
> +import org.eclipse.jetty.server.HttpConnectionFactory;
> import org.eclipse.jetty.server.Server;
> import org.eclipse.jetty.server.ServerConnector;
> import
>
> org.eclipse.jetty.servlet.ServletContextHandler;
>
> @@ -40,7 +43,10 @@ public class MockSolrService
> public MockSolrService()
> {
> server = new Server(new QueuedThreadPool(35));
> - ServerConnector connector = new
>
> ServerConnector(server);
>
> + HttpConfiguration config = new HttpConfiguration();
> + HttpConnectionFactory http1 = new
>
> HttpConnectionFactory(config);
>
> + HTTP2CServerConnectionFactory http2c = new
> HTTP2CServerConnectionFactory(config);
> + ServerConnector connector = new
>
> ServerConnector(server,
>
> http1,
>
> http2c);
> connector.setPort(8188);
> server.addConnector(connector);
> servlet = new SolrServlet();
> @@ -111,6 +117,7 @@ public class MockSolrService
> res.getWriter().printf(Locale.ROOT, "<solr>\n");
> res.getWriter().printf(Locale.ROOT,
>
> "</solr>\n");
>
> res.getWriter().flush();
> }
> ```
>
> - run Junit test
> ```
> ~manifoldcf/connectors/solr% ant run-IT-HSQLDB
> ```
>
> - confirm test-HSQLDB-output/manifoldcf.log
> ```
> ERROR 2023-10-23T22:10:29,902 (Worker thread '24') -
>
> Exception
>
> tossed:
>
> Unhandled Solr exception during indexinghttp://test70.txt
>
> (200):
>
> Error
>
> from server athttp://localhost:8188/solr: Expected mime
>
> type
>
> application/octet-stream but got application/xml.
>
> <result>
>
> <doc name="something"/>
> </result>
>
>
>
> org.apache.manifoldcf.core.interfaces.ManifoldCFException:
>
> Unhandled
>
> Solr
> exception during indexinghttp://test70.txt (200): Error
>
> from
>
> server
>
> at
>
> http://localhost:8188/solr: Expected mime type
>
> application/octet-stream
>
> but
> got application/xml. <result>
> <doc name="something"/>
> </result>
>
> at
>
>
>
> org.apache.manifoldcf.agents.output.solr.HttpPoster.handleSolrException(HttpPoster.java:389)
>
> ~[classes/:?]
> at
>
>
>
> org.apache.manifoldcf.agents.output.solr.HttpPoster.indexPost(HttpPoster.java:544)
>
> ~[classes/:?]
> at
>
>
>
> org.apache.manifoldcf.agents.output.solr.SolrConnector.addOrReplaceDocumentWithException(SolrConnector.java:522)
>
> ~[classes/:?]
> at
>
>
>
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3214)
>
> ~[mcf-agents.jar:?]
> at
>
>
>
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$OutputAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3395)
>
> ~[mcf-agents.jar:?]
> at
>
>
>
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3065)
>
> ~[mcf-agents.jar:?]
> at
>
>
>
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2696)
>
> ~[mcf-agents.jar:?]
> at
>
>
>
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:750)
>
> ~[mcf-agents.jar:?]
> at
>
>
>
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1585)
>
> ~[mcf-pull-agent.jar:?]
> at
>
>
>
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1550)
>
> ~[mcf-pull-agent.jar:?]
> at
>
>
>
> org.apache.manifoldcf.crawler.tests.TestingRepositoryConnector.processDocuments(TestingRepositoryConnector.java:84)
>
> ~[mcf-pull-agent-tests.jar:?]
> at
>
>
>
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:402)
>
> ~[mcf-pull-agent.jar:?]
> ERROR 2023-10-23T22:10:29,902 (Worker thread '10') -
>
> Exception
>
> tossed:
>
> Unhandled Solr exception during indexinghttp://test72.txt
>
> (200):
>
> Error
>
> from server athttp://localhost:8188/solr: Expected mime
>
> type
>
> application/octet-stream but got application/xml.
>
> <result>
>
> <doc name="something"/>
> </result>
> ```
>
> Could you give me some advice?
>
> 2023年10月23日(月) 22:01 Mingchun Zhao<
>
> mingchun.zha...@gmail.com
>
> :
>
> Then, wherever zookeeper is mentioned in
>
> framework/build.xml,
>
> a
>
> reference to those dependencies must also be included.
>
> It looks like zookeeper*.jar was already included in
> connector-test-classpath within kafka/build.xml.
> ```
> <path id="connector-test-classpath">
> <path
>
> refid="mcf-connector-build.connector-test-classpath"/>
>
> <fileset dir="../../lib">
> <include name="zookeeper*.jar"/>
> <include name="kafka-clients*.jar"/>
> <include name="lz4*.jar"/>
> <include name="snappy-java*.jar"/>
> <include name="metrics-core*.jar"/>
> </fileset>
> ```
>
>
> 2023年10月23日(月) 21:50 Karl Wright<daddy...@gmail.com> <daddy...@gmail.com>:
>
>
> Hi,
>
> That just downloads zookeeper. But apparently the
>
> zookeeper
>
> version
>
> required by Kafka now has dependencies of its own.
>
> Otherwise
>
> the
>
> zookeeper
> tests wouldn't fail with linkage errors.
>
> The dependencies need to be identified and added in
>
> several
>
> places. The
> first place is to the download-zookeeper part of the
>
> root
>
> build.xml
>
> script. Then, wherever zookeeper is mentioned in
> framework/build.xml, a
> reference to those dependencies must also be included.
>
> Karl
>
>
> On Mon, Oct 23, 2023 at 8:32 AM Mingchun Zhao <
>
> mingchun.zha...@gmail.com>
>
> wrote:
>
>
> Hi Karl,
>
>
> Mingchun, did you add the jar(s) that the new
>
> zookeeper
>
> needs
>
> to
>
> the
>
> build.xml download section?
>
> Are the following settings correct? Or do you have an
>
> old
>
> version
>
> of
>
> zookeeper-*.jar left in your environment?
>
> ```build.xml
> <target name="download-zookeeper">
> <mkdir dir="lib"/>
> <antcall target="download-via-maven">
> <param name="target" value="lib"/>
> <param name="project-path"
> value="org/apache/zookeeper"/>
> <param name="artifact-version"
>
> value="${zookeeper.version}"/>
>
> <param name="artifact-name"
>
> value="zookeeper"/>
>
> <param name="artifact-type"
>
> value="jar"/>
>
> </antcall>
> ... ...
> ```
>
> Regards,
> Mingchun
>
>
> 2023年10月23日(月) 21:19 Karl Wright<daddy...@gmail.com> <daddy...@gmail.com>:
>
>
> Well, that was interesting.
>
> Didn't get very far, because the dependency updates
>
> committed
>
> broke
>
> Zookeeper again:
>
> [junit] Testcase:
>
>
>
> multiThreadZooKeeperLockTest(org.apache.manifoldcf.core.lockmanager.TestZooKeeperLocks):
>
> Caused an ERROR
> [junit] io/netty/handler/ssl/SslContext
> [junit] java.lang.NoClassDefFoundError:
>
> io/netty/handler/ssl/SslContext
>
> [junit] at
>
>
>
> org.apache.zookeeper.common.ZKConfig.handleBackwardCompatibility(ZKConfig.java:106)
>
> [junit] at
>
>
>
> org.apache.zookeeper.client.ZKClientConfig.handleBackwardCompatibility(ZKClientConfig.java:96)
>
> [junit] at
>
>
> org.apache.zookeeper.common.ZKConfig.init(ZKConfig.java:92)
>
> [junit] at
>
>
> org.apache.zookeeper.common.ZKConfig.<init>(ZKConfig.java:61)
>
> [junit] at
>
>
> org.apache.zookeeper.client.ZKClientConfig.<init>(ZKClientConfig.java:69)
>
> [junit] at
>
>
> org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:643)
>
> [junit] at
>
>
> org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:567)
>
> [junit] at
>
>
> org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:734)
>
> [junit] at
>
>
> org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:448)
>
> [junit] at
>
>
>
> org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.createSession(ZooKeeperConnection.java:74)
>
> [junit] at
>
>
>
> org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.<init>(ZooKeeperConnection.java:66)
>
> It looks like the even newer Zookeeper version has a
>
> newer
>
> dependency
>
> that
>
> isn't being included in the basic classpath, but
>
> should
>
> be.
>
> Mingchun,
>
> did
>
> you add the jar(s) that the new zookeeper needs to
>
> the
>
> build.xml
>
> download
>
> section? If so, can you remind me what they were?
>
> Karl
>
>
> On Mon, Oct 23, 2023 at 8:11 AM Karl Wright<
>
> daddy...@gmail.com
>
> wrote:
>
> I begin to suspect that the problem may be human
>
> error.
>
> If you don't do "ant clean-core-deps; ant
>
> make-core-deps",
>
> but
>
> instead
>
> just use "ant make-core-deps", you could have
>
> incompatible
>
> versions
>
> of
>
> several libraries in your classpath for the tests.
>
> I'll
>
> try
>
> today
>
> to
>
> verify whether that might be happening by trying the
>
> tests
>
> locally
>
> myself.
>
> Karl
>
>
> On Mon, Oct 23, 2023 at 7:57 AM Guylaine BASSETTE
> <guylaine.basse...@francelabs.com> wrote:
>
>
> Hi Karl and Mingchun,
>
> Thanks for your work on the last few issues. I join
>
> you on
>
> this
>
> Solr
>
> testing problem.
>
> That said, we've tested this new connector in our
>
> application,
>
> with a
>
> FileShare job and everything was OK.
>
> I hope and think it's just a problem specific to
>
> the
>
> test.
>
> Missing
>
> updates or incompatible dependencies...
>
> Le 20/10/2023 à 02:58, Mingchun Zhao a écrit :
>
> Hi Karl, Thanks!
>
>
> so I wonder if, once again, there's a problem
>
> with
>
> dependencies
>
> for
>
> the
>
> version of Solr they chose.
>
> I'll take a look at this issue.
>
> 2023年10月20日(金) 9:50 Karl Wright<
>
> daddy...@gmail.com
>
> :
>
> This connector FranceLabs updated. The problem
>
> seems to
>
> occur
>
> at a
>
> basic
>
> level during http2 communication, so I wonder if,
>
> once
>
> again,
>
> there's a
>
> problem with dependencies for the version of Solr
>
> they
>
> chose.
>
> Karl
>
>
> On Thu, Oct 19, 2023 at 8:32 PM Mingchun Zhao<
>
> mingchun.zha...@gmail.com>
>
> wrote:
>
>
> About the test "SolrCrawlHSQLDBIT" failure, it
>
> seems
>
> that
>
> "IO
>
> exception
>
> during indexinghttp://test58.txt:
>
> frame_size_error/invalid_frame_length"
>
> error is occurring on the ManifoldCF side.
>
> - command:
> ```
> manifoldcf/connectors/solr% ant run-IT-HSQLDB
>
> run-IT-HSQLDB:
> [junit] Testsuite:
>
>
> org.apache.manifoldcf.agents.output.solr.tests.SolrCrawlHSQLDBIT
>
> ... ...
> ```
>
> - I checked
>
> "connectors/solr/test-HSQLDB-output/manifoldcf.log":
>
> ```
> WARN 2023-10-20T09:14:56,635 (Worker thread
>
> '18') -
>
> IO
>
> exception
>
> during
>
> indexinghttp://test58.txt:
>
> frame_size_error/invalid_frame_length
>
> java.io.IOException:
>
> frame_size_error/invalid_frame_length
>
> at
>
> org.eclipse.jetty.http2.HTTP2Session.toFailure(HTTP2Session.java:566)
>
> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
> org.eclipse.jetty.http2.HTTP2Session.access$2700(HTTP2Session.java:80)
>
> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
>
>
> org.eclipse.jetty.http2.HTTP2Session$StreamsState.onSessionFailure(HTTP2Session.java:1857)
>
> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
>
>
> org.eclipse.jetty.http2.HTTP2Session$StreamsState.access$400(HTTP2Session.java:1436)
>
> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
>
>
> org.eclipse.jetty.http2.HTTP2Session.onSessionFailure(HTTP2Session.java:511)
>
> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
>
>
> org.eclipse.jetty.http2.HTTP2Session.onConnectionFailure(HTTP2Session.java:506)
>
> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
>
>
> org.eclipse.jetty.http2.parser.Parser$Listener$Wrapper.onConnectionFailure(Parser.java:414)
>
> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
>
>
> org.eclipse.jetty.http2.HTTP2Connection$ParserListener.onConnectionFailure(HTTP2Connection.java:397)
>
> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
>
>
> org.eclipse.jetty.http2.parser.BodyParser.notifyConnectionFailure(BodyParser.java:223)
>
> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
>
>
> org.eclipse.jetty.http2.parser.BodyParser.connectionFailure(BodyParser.java:215)
>
> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
> org.eclipse.jetty.http2.parser.Parser.connectionFailure(Parser.java:209)
>
> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
> org.eclipse.jetty.http2.parser.Parser.parseHeader(Parser.java:155)
> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
> org.eclipse.jetty.http2.parser.Parser.parse(Parser.java:121)
>
> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
>
>
> org.eclipse.jetty.http2.HTTP2Connection$HTTP2Producer.produce(HTTP2Connection.java:261)
>
> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
>
>
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362)
>
> ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
>
>
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186)
>
> ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
>
>
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
>
> ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
>
>
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137)
>
> ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
>
> org.eclipse.jetty.http2.HTTP2Connection.produce(HTTP2Connection.java:183)
>
> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
>
>
> org.eclipse.jetty.http2.HTTP2Connection.onFillable(HTTP2Connection.java:138)
>
> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
>
>
> org.eclipse.jetty.http2.HTTP2Connection$FillableCallback.succeeded(HTTP2Connection.java:361)
>
> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at org.eclipse.jetty.io
>
> .FillInterest.fillable(FillInterest.java:105)
>
> ~[jetty-io-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at org.eclipse.jetty.io
>
> .ChannelEndPoint$1.run(ChannelEndPoint.java:104)
>
> ~[jetty-io-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
>
>
> org.eclipse.jetty.util.thread.Invocable.invokeNonBlocking(Invocable.java:69)
>
> ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
>
>
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.invokeTask(EatWhatYouKill.java:350)
>
> ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
>
>
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:305)
>
> ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
>
>
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
>
> ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
>
>
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137)
>
> ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622]
>
> at
>
>
>
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:271)
>
> ~[solr-solrj-9.1.0.jar:9.1.0
>
> aa4f3d98ab19c201e7f3c74cd14c99174148616d
>
> -
>
> ishan - 2022-11-11 13:00:47]
> at
>
>
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>
> ~[?:?]
> at
>
>
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>
> ~[?:?]
> at java.lang.Thread.run(Thread.java:829) ~[?:?]
> ```
>
> 2023年10月19日(木) 20:37 Mingchun
> Zhao<mingchun.zha...@gmail.com
>
> :
>
> I'm having trouble with the Solr test `ant
>
> run-IT-HSQLDB`.
>
> It's been over an hour since I ran the test and
>
> it
>
> still
>
> doesn't
>
> finish.
>
> It seems that the problem is probably due to
>
> the
>
> Manifold
>
> job
>
> not
>
> finishing.
> Do you have any ideas or advice? Thanks!
>
> ```
> ~/ManifoldCF/manifoldcf/connectors/solr% ant
>
> run-IT-HSQLDB
>
> Buildfile:
>
>
> /Users/zhaomingchun/ManifoldCF/manifoldcf/connectors/solr/build.xml
>
> ... ...
> run-IT-HSQLDB:
> [junit] Testsuite:
>
>
> org.apache.manifoldcf.agents.output.solr.tests.SolrCrawlHSQLDBIT
>
> [junit] Configuration file successfully
>
> read
>
> [junit] [main] INFO
>
> org.eclipse.jetty.util.log
>
> -
>
> Logging
>
> initialized
>
> @7416ms to org.eclipse.jetty.util.log.Slf4jLog
> [junit] [main] INFO
>
> org.eclipse.jetty.server.Server -
>
> jetty-9.4.48.v20220622; built:
>
> 2022-06-21T20:42:25.880Z;
>
> git:
>
> 6b67c5719d1f4371b33655ff2d047d24e171e49a; jvm
>
> 11.0.11+9
>
> [junit] [main] INFO
>
> org.eclipse.jetty.server.session
>
> -
>
> DefaultSessionIdManager workerName=node0
> [junit] [main] INFO
>
> org.eclipse.jetty.server.session
>
> -
>
> No
>
> SessionScavenger set, using defaults
> [junit] [main] INFO
>
> org.eclipse.jetty.server.session
>
> -
>
> node0
>
> Scavenging every 600000ms
> [junit] [main] INFO
>
> org.eclipse.jetty.server.handler.ContextHandler -
>
> Started o.e.j.w.WebAppContext@4bdc8b5d
>
> {ManifoldCF
>
> Crawler
>
> Interface,/mcf-crawler-ui,file:///private/var/folders/zh/mx4q_qh93cv6jtp13ht8b1fr0000gn/T/jetty-0_0_0_0-8346-mcf-crawler-ui_war-_mcf-crawler-ui-any-17189990303852051874/webapp/,AVAILABLE}{/Users/zhaomingchun/ManifoldCF/manifoldcf/dist/web/war/mcf-crawler-ui.war}
>
> [junit] [main] INFO
>
> org.eclipse.jetty.server.handler.ContextHandler -
>
> Started o.e.j.w.WebAppContext@3bcd426c
>
> {ManifoldCF
>
> Authorities
>
> API
>
> Webapp,/mcf-authority-service,file:///private/var/folders/zh/mx4q_qh93cv6jtp13ht8b1fr0000gn/T/jetty-0_0_0_0-8346-mcf-authority-service_war-_mcf-authority-service-any-1541416273384984853/webapp/,AVAILABLE}{/Users/zhaomingchun/ManifoldCF/manifoldcf/dist/web/war/mcf-authority-service.war}
>
> [junit] Creating mock service
> [junit] [main] INFO
>
> org.eclipse.jetty.server.handler.ContextHandler -
>
> Started o.e.j.w.WebAppContext@5f14a673
>
> {ManifoldCF
>
> General
>
> API
>
>
> Webapp,/mcf-api-service,file:///private/var/folders/zh/mx4q_qh93cv6jtp13ht8b1fr0000gn/T/jetty-0_0_0_0-8346-mcf-api-service_war-_mcf-api-service-any-12421572948842353797/webapp/,AVAILABLE}{/Users/zhaomingchun/ManifoldCF/manifoldcf/dist/web/war/mcf-api-service.war}
>
> [junit] Mock service created
> [junit] [main] INFO
>
> org.eclipse.jetty.server.AbstractConnector -
>
> Started ServerConnector@70325d20{HTTP/1.1,
>
> (http/1.1)}{
>
> 0.0.0.0:8346}
>
> [junit] [main] INFO
>
> org.eclipse.jetty.server.Server -
>
> Started
>
> @9616ms
>
> [junit] [main] INFO
>
> org.eclipse.jetty.server.Server -
>
> jetty-9.4.48.v20220622; built:
>
> 2022-06-21T20:42:25.880Z;
>
> git:
>
> 6b67c5719d1f4371b33655ff2d047d24e171e49a; jvm
>
> 11.0.11+9
>
> [junit] [main] INFO
>
> org.eclipse.jetty.server.session
>
> -
>
> DefaultSessionIdManager workerName=node0
> [junit] [main] INFO
>
> org.eclipse.jetty.server.session
>
> -
>
> No
>
> SessionScavenger set, using defaults
> [junit] [main] INFO
>
> org.eclipse.jetty.server.session
>
> -
>
> node0
>
> Scavenging every 600000ms
> [junit] [main] INFO
>
> org.eclipse.jetty.server.handler.ContextHandler -
>
> Started o.e.j.s.ServletContextHandler@7808f638
>
> {/solr,null,AVAILABLE}
>
> [junit] [main] INFO
>
> org.eclipse.jetty.server.AbstractConnector -
>
> Started ServerConnector@2ab5afc7{HTTP/1.1,
>
> (http/1.1)}{
>
> 0.0.0.0:8188}
>
> [junit] [main] INFO
>
> org.eclipse.jetty.server.Server -
>
> Started
>
> @9626ms
>
> [junit] [main] INFO
>
> org.eclipse.jetty.server.AbstractConnector -
>
> Stopped ServerConnector@2ab5afc7{HTTP/1.1,
>
> (http/1.1)}{
>
> 0.0.0.0:8188}
>
> [junit] [main] INFO
>
> org.eclipse.jetty.server.session
>
> -
>
> node0
>
> Stopped
>
> scavenging
> [junit] [main] INFO
>
> org.eclipse.jetty.server.handler.ContextHandler -
>
> Stopped o.e.j.s.ServletContextHandler@7808f638
>
> {/solr,null,STOPPED}
>
> ```
>
> 2023年10月19日(木) 20:05 Mingchun Zhao<
>
> mingchun.zha...@gmail.com
>
> :
>
> Hi Karl,
>
> I've tried to update Kafka and its
>
> dependencies
>
> to
>
> the
>
> latest
>
> version
>
> including zookeeper,
> and confirmed tha kafka test run-IT-HSQLDB
>
> passed
>
> as
>
> below:
>
> ```
> ~manifoldcf% cd connectors/kafka
> ~manifoldcf/connectors/kafka/% ant
>
> run-IT-HSQLDB
>
> BUILD SUCCESSFUL
> Total time: 1 minute 19 seconds
> ```
>
> Also, I prepared a PR for this
> issue:https://github.com/apache/manifoldcf/pull/155
>
> 2023年10月19日(木) 7:12 Karl Wright<
>
> daddy...@gmail.com>:
>
> Hi,
>
> It looks like the latest release of Kafka is
>
> 3.6.0.
>
> I'd try setting that version in the pom for
>
> connectors/kafka
>
> and
>
> doing
>
> mvn
> install. Then you can see what dependencies
>
> it
>
> wants
>
> by:
>
> mvn dependency:tree
>
> It may be that Kafka no longer even requires
>
> zookeeper
>
> -
>
> I
>
> didn't
>
> find
>
> it
>
> in a cursory inspection. But the
>
> dependency:tree
>
> would
>
> be
>
> the
>
> final
>
> word.
>
> Karl
>
>
> On Sat, Oct 14, 2023 at 2:17 AM Mingchun
>
> Zhao <
>
> mingchun.zha...@gmail.com
>
> wrote:
>
>
> Kalr, Thanks!
> Though I'm not familiar with kafka, I'll try
>
> to
>
> find
>
> out
>
> what's
>
> causing the
>
> error as much as possible.
>
> Kind Regards,
> Mingchun
>
>
> 2023年10月14日(土) 14:07 Karl Wright<
>
> daddy...@gmail.com
>
> :
>
> Yes, this seems to be something related to
>
> zookeeper
>
> update
>
> and
>
> the
>
> Kafka
>
> library version we're using.
>
> Someone will need to dig into what is going
>
> wrong
>
> here
>
> before
>
> we
>
> can
>
> release. I don't know how widely used the
>
> kafka
>
> connector is
>
> but
>
> if
>
> it
>
> is
>
> lightly used we can perhaps not distribute
>
> the
>
> connector
>
> any
>
> longer. But
>
> that would be a last choice.
>
> Karl
>
>
> On Fri, Oct 13, 2023 at 12:12 PM Mingchun
>
> Zhao
>
> <
>
> mingchun.zha...@gmail.com>
>
> wrote:
>
>
> By applying r1912939, I was able to
>
> confirm
>
> that
>
> the
>
> kafka
>
> test
>
> compile
>
> error has disappeared when running `ant
>
> test`.
>
> Thanks, Karl!
>
> However, the following error occurred on
>
> subsequent
>
> test
>
> runs.
>
> ```
> [junit] Testsuite:
>
>
> org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT
>
> [junit] Tests run: 1, Failures: 0, Errors:
>
> 1,
>
> Skipped:
>
> 0,
>
> Time
>
> elapsed:
>
> 0 sec
> [junit]
> [junit] Testcase:
>
>
> org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT:sanityCheck:
>
> Caused an ERROR
> [junit] Forked Java VM exited
>
> abnormally.
>
> Please
>
> note
>
> the
>
> time
>
> in
>
> the
>
> report does not reflect the time until the
>
> VM
>
> exit.
>
> [junit]
>
> junit.framework.AssertionFailedError:
>
> Forked
>
> Java
>
> VM
>
> exited
>
> abnormally. Please note the time in the
>
> report
>
> does
>
> not
>
> reflect
>
> the
>
> time
>
> until the VM exit.
> [junit] at
>
>
> jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown
>
> Source)
>
> [junit] at
>
>
>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> [junit] at
>
>
> jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown
>
> Source)
>
> [junit] at
>
>
>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> [junit] at
>
>
> jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown
>
> Source)
>
> [junit] at
>
>
>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> [junit] at
>
>
> jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown
>
> Source)
>
> [junit] at
>
>
>
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> [junit]
> [junit]
>
> BUILD FAILED
>
>
> /Users/zhaomingchun/ManifoldCF/manifoldcf/build.xml:517:
>
> The
>
> following
>
> error occurred while executing this line:
>
>
> /Users/zhaomingchun/ManifoldCF/manifoldcf/build.xml:471:
>
> The
>
> following
>
> error occurred while executing this line:
>
>
> /Users/zhaomingchun/ManifoldCF/manifoldcf/dist/connector-build.xml:1102:
>
> Test
>
> org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT
>
> failed
>
> (crashed)
> ```
>
>
> 2023年10月13日(金) 21:56 Karl Wright<
>
> daddy...@gmail.com
>
> :
>
> r1912939 fixes this but I need to spin a
>
> new
>
> RC.
>
> Karl
>
>
> On Fri, Oct 13, 2023 at 8:46 AM Karl
>
> Wright <
>
> daddy...@gmail.com>
>
> wrote:
>
> Yes I get the same thing; a test needs
>
> to
>
> be
>
> updated.
>
> [javac]
>
>
> C:\wip\mcf\release-2.26-branch\connectors\kafka\connector\src\test\java\org\apache\manifoldcf\agents\output\kafka\ZooKeeperLocal.java:45:
>
> error: unreported exception
>
> AdminServerException;
>
> must
>
> be
>
> caught or
>
> declared to be thrown
> [javac]
>
> zooKeeperServer.runFromConfig(configuration);
>
> [javac]
>
> Karl
>
>
> On Fri, Oct 13, 2023 at 8:35 AM Karl
>
> Wright
>
> <
>
> daddy...@gmail.com>
>
> wrote:
>
> There was a Zookeeper dependency change
>
> this
>
> release. I
>
> wonder if
>
> there
>
> is a test that needs to be updated.
>
> Let
>
> me
>
> try
>
> and
>
> see.
>
> Karl
>
>
> On Fri, Oct 13, 2023 at 4:51 AM
>
> Piergiorgio
>
> Lucidi
>
> <
>
> piergior...@apache.org> wrote:
>
>
> Hi Mingchun,
>
> thank you for your message and I was
>
> trying
>
> to
>
> build
>
> ManifoldCF
>
> using
>
> OpenJDK 17 so probably in the future
>
> for
>
> supporting
>
> this
>
> version
>
> of
>
> Java
>
> we
> should include Jaxb libraries as well.
>
> The build is ok now and I can compile
>
> and
>
> package
>
> everything
>
> correctly.
>
> Unfortunately executing tests I have
>
> the
>
> following
>
> error:
>
> compile-tests:
> [javac] Compiling 1 source file
>
> to
>
> /Users/piergiorgiolucidi/Downloads/apache-manifoldcf-2.26/connectors/kafka/build/connector-tests/classes
>
> [javac]
>
>
>
> /Users/piergiorgiolucidi/Downloads/apache-manifoldcf-2.26/connectors/kafka/connector/src/test/java/org/apache/manifoldcf/agents/output/kafka/ZooKeeperLocal.java:45:
>
> error: unreported exception
>
> AdminServerException;
>
> must
>
> be
>
> caught
>
> or
>
> declared to be thrown
> [javac]
>
> zooKeeperServer.runFromConfig(configuration);
>
> [javac]
>
> ^
>
> [javac] 1 error
>
> BUILD FAILED
>
>
> /Users/piergiorgiolucidi/Downloads/apache-manifoldcf-2.26/build.xml:497:
>
> The following error occurred while
>
> executing
>
> this
>
> line:
>
> /Users/piergiorgiolucidi/Downloads/apache-manifoldcf-2.26/build.xml:471:
>
> The following error occurred while
>
> executing
>
> this
>
> line:
>
> /Users/piergiorgiolucidi/Downloads/apache-manifoldcf-2.26/dist/connector-build.xml:720:
>
> Compile failed; see the compiler error
>
> output
>
> for
>
> details.
>
> Any ideas?
>
> Thanks.
>
> Cheers,
> PG
>
>
>
> Il giorno gio 12 ott 2023 alle ore
>
> 10:39
>
> Mingchun
>
> Zhao <
>
> mingchun.zha...@gmail.com> ha
>
> scritto:
>
> Hi Piergiorgio,
>
> FYI, Allow me to share the java and
>
> ant
>
> versions
>
> and
>
> the
>
> build
>
> steps
>
> I
>
> ran
>
> in my environment.
>
> ```
> $ java --version
> openjdk 11.0.11 2021-04-20
> OpenJDK Runtime Environment
>
> AdoptOpenJDK-11.0.11+9
>
> (build
>
> 11.0.11+9)
>
> OpenJDK 64-Bit Server VM
>
> AdoptOpenJDK-11.0.11+9
>
> (build
>
> 11.0.11+9,
>
> mixed
>
> mode)
>
> $ ant -version
> Apache Ant(TM) version 1.10.0
>
> compiled
>
> on
>
> December
>
> 27
>
> 2016
>
> ```
> ```
> ant clean
> ant clean-deps
> ant clean-core-deps
> ant make-core-deps
> ant make-deps
> ant build
> ```
>
> Regards,
> Mingchun
>
> 2023年10月12日(木) 17:32 Piergiorgio
>
> Lucidi <
>
> piergior...@apache.org
>
> :
>
> Hi folks,
>
> it seems that I can't compile the
>
> CswsConnector:
>
> [javac] public List<? extends
>
> AttributeGroup>
>
> getAttributeGroups()
>
> [javac] ^
> [javac] symbol: class
>
> AttributeGroup
>
> [javac] location: class
>
> CswsConnector.ObjectInformation
>
> [javac]
>
>
>
> /Volumes/BackupPJ/ManifoldCF-release/apache-manifoldcf-2.26/connectors/csws/connector/src/main/java/org/apache/manifoldcf/crawler/connectors/csws/CswsConnector.java:3966:
>
> error: cannot find symbol
> [javac] public
>
> NodePermissions
>
> getPermissions()
>
> [javac] ^
> [javac] symbol: class
>
> NodePermissions
>
> [javac] location: class
>
> CswsConnector.ObjectInformation
>
> [javac] 100 errors
> [javac] 1 warning
> [javac] only showing the
>
> first
>
> 100
>
> errors,
>
> of
>
> 123
>
> total;
>
> use
>
> -Xmaxerrs
>
> if you would like to see more
>
> BUILD FAILED
>
>
> /Volumes/BackupPJ/ManifoldCF-release/apache-manifoldcf-2.26/build.xml:489:
>
> The following error occurred while
>
> executing
>
> this
>
> line:
>
>
> /Volumes/BackupPJ/ManifoldCF-release/apache-manifoldcf-2.26/build.xml:471:
>
> The following error occurred while
>
> executing
>
> this
>
> line:
>
> /Volumes/BackupPJ/ManifoldCF-release/apache-manifoldcf-2.26/dist/connector-build.xml:686:
>
> Compile failed; see the compiler
>
> error
>
> output
>
> for
>
> details.
>
> Do you have any ideas?
>
> Thanks,
> PG
>
> Il giorno gio 12 ott 2023 alle ore
>
> 09:03
>
> Guylaine
>
> BASSETTE <
>
> guylaine.basse...@francelabs.com>
>
> ha
>
> scritto:
>
> Hi,
>
> +1 from France Labs
>
> Regards,
> Guylaine
>
> France Labs – Your knowledge, now
> Datafari Enterprise Search –
>
> Découvrez
>
> la
>
> version
>
> 5 /
>
> Discover
>
> our
>
> version
>
> 5www.datafari.com <
>
> http://www.datafari.com>
>
> Le 11/10/2023 à 23:12, Furkan
>
> KAMACI
>
> a
>
> écrit
>
> :
>
> Hi,
>
> +1!
>
> Kind regards,
> Furkan Kamaci
>
> On Wed, Oct 11, 2023 at 11:13 PM
>
> Mingchun
>
> Zhao<
>
> mingchun.zha...@gmail.com
>
> wrote:
>
>
> Hi,
> Ran all the tests.
> +1 from me.
>
> Kind regards,
> Mingchun
>
> 2023年10月12日(木) 4:29 Cihad Guzel<
>
> cguz...@gmail.com
>
> :
>
> Hi,
>
> +1 from me
>
> Regards,
> Cihad Guzel
>
> 11 Eki 2023 Çar, saat 15:08
>
> tarihinde
>
> Karl
>
> Wright<
>
> daddy...@gmail.com
>
> şunu
> yazdı:
>
>
> Please vote on whether to
>
> release
>
> Apache
>
> ManifoldCF
>
> 2.26,
>
> RC0.
>
> This release is the first
>
> release
>
> that
>
> requires
>
> at
>
> least
>
> Java
>
> 11,
>
> and
>
> it
>
> also includes a new CSV
>
> connector
>
> along
>
> with
>
> support
>
> for
>
> Solr
>
> 9.
>
> The
>
> release artifact can be found
>
> at:
>
> https://dist.apache.org/repos/dist/dev/manifoldcf/apache-manifoldcf-2.26
>
> ,
>
> and there is a release tag also
>
> at
>
> https://svn.apache.org/repos/asf/manifoldcf/tags/release-2.26-RC0
>
> .
>
> Karl
>
>
> --
> Piergiorgio
>
>
> --
> Piergiorgio
>
>
> --
> Cordialement,
> Guylaine
>
> France Labs – Your knowledge, now
> Datafari Enterprise Search – Découvrez la version
>
> 5 /
>
> Discover
>
> our
>
> version 5www.datafari.com <http://www.datafari.com>
> <http://www.datafari.com>
>
> --
> Cordialement,
> Guylaine
>
> France Labs – Your knowledge, now
> Datafari Enterprise Search – Découvrez la version 5 /
>
> Discover
>
> our
>
> version
>
> 5www.datafari.com <http://www.datafari.com> <http://www.datafari.com>
>
> --
> Cordialement,
> Guylaine
>
> France Labs – Your knowledge, now
> Datafari Enterprise Search – Découvrez la version 5 / Discover our
>
> version
>
> 5www.datafari.com <http://www.datafari.com> <http://www.datafari.com>
>
> --
> Cordialement,
> Guylaine
>
> France Labs – Your knowledge, now
> Datafari Enterprise Search – Découvrez la version 5 / Discover our
> version 5www.datafari.com <http://www.datafari.com> <http://www.datafari.com>
>
> --
> Cordialement,
> Guylaine
>
> France Labs – Your knowledge, now
> Datafari Enterprise Search – Découvrez la version 5 / Discover our version
> 5
> www.datafari.com
>
