I ran the ant test using Guylaine's patch and the Kafka IT test passed! However, the Solr IT test failure still occurs. ``` run-IT-HSQLDB: [junit] Testsuite: org.apache.manifoldcf.agents.output.solr.tests.SolrCrawlHSQLDBIT [junit] Configuration file successfully read [junit] [main] INFO org.eclipse.jetty.util.log - Logging initialized @6974ms to org.eclipse.jetty.util.log.Slf4jLog [junit] [main] INFO org.eclipse.jetty.server.Server - jetty-9.4.48.v20220622; built: 2022-06-21T20:42:25.880Z; git: 6b67c5719d1f4371b33655ff2d047d24e171e49a; jvm 11.0.11+9 [junit] [main] INFO org.eclipse.jetty.server.session - DefaultSessionIdManager workerName=node0 [junit] [main] INFO org.eclipse.jetty.server.session - No SessionScavenger set, using defaults [junit] [main] INFO org.eclipse.jetty.server.session - node0 Scavenging every 660000ms [junit] [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.w.WebAppContext@4565a70a{ManifoldCF Crawler Interface,/mcf-crawler-ui,file:///private/var/folders/zh/mx4q_qh93cv6jtp13ht8b1fr0000gn/T/jetty-0_0_0_0-8346-mcf-crawler-ui_war-_mcf-crawler-ui-any-3250496980318085250/webapp/,AVAILABLE}{/Users/zhaomingchun/ManifoldCF/guylaine/manifoldcf/dist/web/war/mcf-crawler-ui.war} [junit] [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.w.WebAppContext@a0a9fa5{ManifoldCF Authorities API Webapp,/mcf-authority-service,file:///private/var/folders/zh/mx4q_qh93cv6jtp13ht8b1fr0000gn/T/jetty-0_0_0_0-8346-mcf-authority-service_war-_mcf-authority-service-any-6182863812882423778/webapp/,AVAILABLE}{/Users/zhaomingchun/ManifoldCF/guylaine/manifoldcf/dist/web/war/mcf-authority-service.war} [junit] Creating mock service [junit] Mock service created [junit] [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.w.WebAppContext@15723761{ManifoldCF General API Webapp,/mcf-api-service,file:///private/var/folders/zh/mx4q_qh93cv6jtp13ht8b1fr0000gn/T/jetty-0_0_0_0-8346-mcf-api-service_war-_mcf-api-service-any-14231646742487500933/webapp/,AVAILABLE}{/Users/zhaomingchun/ManifoldCF/guylaine/manifoldcf/dist/web/war/mcf-api-service.war} [junit] [main] INFO org.eclipse.jetty.server.AbstractConnector - Started ServerConnector@6f63c44f{HTTP/1.1, (http/1.1)}{0.0.0.0:8346} [junit] [main] INFO org.eclipse.jetty.server.Server - Started @9062ms [junit] [main] INFO org.eclipse.jetty.server.Server - jetty-9.4.48.v20220622; built: 2022-06-21T20:42:25.880Z; git: 6b67c5719d1f4371b33655ff2d047d24e171e49a; jvm 11.0.11+9 [junit] [main] INFO org.eclipse.jetty.server.session - DefaultSessionIdManager workerName=node0 [junit] [main] INFO org.eclipse.jetty.server.session - No SessionScavenger set, using defaults [junit] [main] INFO org.eclipse.jetty.server.session - node0 Scavenging every 660000ms [junit] [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@6b2e0f78{/solr,null,AVAILABLE} [junit] [main] INFO org.eclipse.jetty.server.AbstractConnector - Started ServerConnector@3ec2ecea{HTTP/1.1, (http/1.1)}{0.0.0.0:8188} [junit] [main] INFO org.eclipse.jetty.server.Server - Started @9071ms [junit] [main] INFO org.eclipse.jetty.server.AbstractConnector - Stopped ServerConnector@3ec2ecea{HTTP/1.1, (http/1.1)}{0.0.0.0:8188} [junit] [main] INFO org.eclipse.jetty.server.session - node0 Stopped scavenging [junit] [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Stopped o.e.j.s.ServletContextHandler@6b2e0f78{/solr,null,STOPPED} ```
2023年10月27日(金) 0:35 Guylaine BASSETTE <guylaine.basse...@francelabs.com>: > Hi Mingchun, > > If you want, you can access my repository directly: > https://github.com/guylaine-bassette/manifoldcf > > I have made a Fork of Github ManifoldCF. Use the branch: > Fix-kafka-IT-tests. It is up to date with the last trunk commit. > > Le 26/10/2023 à 17:13, Mingchun Zhao a écrit : > > Sorry, there was a typo: track -> trunk > > > > 2023年10月27日(金) 0:12 Mingchun Zhao<mingchun.zha...@gmail.com>: > > > >> Hi Guylaine, Thanks! > >> Was your patch based on the latest track? Unfortunately, I was unable to > >> apply your patch in my environment. > >> > >> 2023年10月26日(木) 20:44 Guylaine BASSETTE<guylaine.basse...@francelabs.com > >: > >> > >>> Hello Mingchun, > >>> > >>> As mentioned in my previous e-mail to Karl, my patch contained other > >>> files to be updated. I don't know why they don't have been taken into > >>> account, but would you be interested in trying my patch? I have fixed > the > >>> spots I had missed... (joint with this mail) > >>> > >>> With these modifications I went through all tests of kafka. > >>> Le 26/10/2023 à 03:47, Mingchun Zhao a écrit : > >>> > >>> Hi there, Allow me to share my `ant test` result using the latest > Github > >>> trunk. > >>> I got a lot of `Broker may not be available` warnings in the Kafka IT > test > >>> and it eventually failed. > >>> > >>> ``` > >>> ... > >>> [junit] [kafka-producer-network-thread | producer-3] INFO > >>> org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-3] > >>> Node 0 disconnected. > >>> [junit] [kafka-producer-network-thread | producer-3] WARN > >>> org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-3] > >>> Connection to node 0 (/192.168.10.103:9092) could not be established. > >>> Broker may not be available. > >>> [junit] [kafka-producer-network-thread | producer-1] INFO > >>> org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] > >>> Node 0 disconnected. > >>> [junit] [kafka-producer-network-thread | producer-1] WARN > >>> org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] > >>> Connection to node 0 (/192.168.10.103:9092) could not be established. > >>> Broker may not be available. > >>> [junit] [kafka-producer-network-thread | producer-2] INFO > >>> org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-2] > >>> Node 0 disconnected. > >>> [junit] [kafka-producer-network-thread | producer-2] WARN > >>> org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-2] > >>> Connection to node 0 (/192.168.10.103:9092) could not be established. > >>> Broker may not be available. > >>> [junit] [kafka-producer-network-thread | producer-3] INFO > >>> org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-3] > >>> Node 0 disconnected. > >>> [junit] [kafka-producer-network-thread | producer-3] WARN > >>> org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-3] > >>> Connection to node 0 (/192.168.10.103:9092) could not be established. > >>> Broker may not be available. > >>> [junit] ------------- ---------------- --------------- > >>> [junit] Testcase: > >>> > sanityCheck(org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT): > >>> Caused an ERROR > >>> [junit] ManifoldCF did not terminate in the allotted time of > 120000 > >>> milliseconds > >>> [junit] org.apache.manifoldcf.core.interfaces.ManifoldCFException: > >>> ManifoldCF did not terminate in the allotted time of 120000 > milliseconds > >>> [junit] at > >>> > org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT.waitJobInactive(APISanityHSQLDBIT.java:289) > >>> [junit] at > >>> > org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT.sanityCheck(APISanityHSQLDBIT.java:177) > >>> [junit] at > >>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > >>> Method) > >>> [junit] at > >>> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > >>> [junit] at > >>> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >>> [junit] > >>> [junit] > >>> > >>> BUILD FAILED > >>> /Users/zhaomingchun/ManifoldCF/manifoldcf/build.xml:517: The following > >>> error occurred while executing this line: > >>> /Users/zhaomingchun/ManifoldCF/manifoldcf/build.xml:471: The following > >>> error occurred while executing this line: > >>> > /Users/zhaomingchun/ManifoldCF/manifoldcf/dist/connector-build.xml:1104: > >>> Test org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT failed > >>> > >>> Total time: 38 minutes 13 seconds > >>> ``` > >>> > >>> > >>> 2023年10月26日(木) 5:42 Karl Wright<daddy...@gmail.com> < > daddy...@gmail.com>: > >>> > >>> > >>> I see you have committed this. You missed a few spots; the extra jar > was > >>> mentioned in multiple places. I committed another fix to correct that. > >>> > >>> Karl > >>> > >>> > >>> On Wed, Oct 25, 2023 at 10:46 AM Guylaine BASSETTE< > guylaine.basse...@francelabs.com> wrote: > >>> > >>> > >>> Hello Karl, > >>> > >>> Thank you very much for this update! I have tested your suggestions and > >>> kafka ITs tests ended successfully. :-) > >>> > >>> Here you can find the patch. > >>> > >>> My only doubt is this warning I had in some parts of the kafka test: > >>> > >>> ``` > >>> > >>> [junit] [Controller-0-to-broker-0-send-thread] INFO > >>> org.apache.kafka.clients.NetworkClient - [Controller id=0, > >>> targetBrokerId=0] Node 0 disconnected. > >>> [junit] [Controller-0-to-broker-0-send-thread] WARN > >>> org.apache.kafka.clients.NetworkClient - [Controller id=0, > >>> targetBrokerId=0] Connection to node 0 (guylaine-virtual-machine/ > 127.0.1.1:9092) could not be established. Broker may not be available. > >>> [junit] [Controller-0-to-broker-0-send-thread] WARN > >>> kafka.controller.RequestSendThread - [RequestSendThread controllerId=0] > >>> Controller 0's connection to broker guylaine-virtual-machine:9092 (id: > 0 > >>> rack: null) was unsuccessful > >>> [junit] java.io.IOException: Connection to > >>> guylaine-virtual-machine:9092 (id: 0 rack: null) failed. > >>> [junit] at > >>> > >>> > >>> > org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70) > >>> > >>> [junit] at > >>> > >>> > >>> > kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:298) > >>> > >>> [junit] at > >>> > >>> > >>> > kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:251) > >>> > >>> [junit] at > >>> > >>> > >>> > org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:130) > >>> > >>> [junit] [Controller-0-to-broker-0-send-thread] INFO > >>> org.apache.kafka.clients.NetworkClient - [Controller id=0, > >>> targetBrokerId=0] Client requested connection close from node 0 > >>> [junit] [controller-event-thread] INFO state.change.logger - > >>> [Controller id=0 epoch=2] Sending LeaderAndIsr request to broker 0 > with 1 > >>> become-leader and 0 become-follower partitions > >>> ``` > >>> > >>> I have also ran an "ant test". It run core-framework and ITs tests > until > >>> mongoDB connector with this fail: > >>> ``` > >>> [junit] Testcase: > >>> > >>> > >>> > sanityCheck(org.apache.manifoldcf.agents.output.mongodboutput.tests.APISanityHSQLDBIT): > >>> > >>> Caused an ERROR > >>> [junit] Could not start process: <EOF> > >>> [junit] java.lang.RuntimeException: Could not start process: <EOF> > >>> [junit] at > >>> > >>> > >>> > de.flapdoodle.embed.mongo.AbstractMongoProcess.onAfterProcessStart(AbstractMongoProcess.java:81) > >>> > >>> [junit] at > >>> > >>> > >>> > de.flapdoodle.embed.process.runtime.AbstractProcess.<init>(AbstractProcess.java:115) > >>> > >>> [junit] at > >>> > >>> > >>> > de.flapdoodle.embed.mongo.AbstractMongoProcess.<init>(AbstractMongoProcess.java:54) > >>> > >>> [junit] at > >>> de.flapdoodle.embed.mongo.MongodProcess.<init>(MongodProcess.java:50) > >>> [junit] at > >>> > >>> > >>> > de.flapdoodle.embed.mongo.MongodExecutable.start(MongodExecutable.java:44) > >>> > >>> [junit] at > >>> > >>> > >>> > de.flapdoodle.embed.mongo.MongodExecutable.start(MongodExecutable.java:34) > >>> > >>> [junit] at > >>> > de.flapdoodle.embed.process.runtime.Executable.start(Executable.java:109) > >>> [junit] at > >>> > >>> > >>> > org.apache.manifoldcf.agents.output.mongodboutput.tests.BaseITHSQLDB.setUpMongoDB(BaseITHSQLDB.java:72) > >>> > >>> [junit] at > >>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > >>> Method) > >>> [junit] at > >>> > >>> > >>> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > >>> > >>> [junit] at > >>> > >>> > >>> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >>> > >>> [junit] > >>> [junit] > >>> [junit] Testcase: > >>> > >>> > >>> > sanityCheck(org.apache.manifoldcf.agents.output.mongodboutput.tests.APISanityHSQLDBIT): > >>> > >>> Caused an ERROR > >>> [junit] null > >>> [junit] java.lang.NullPointerException > >>> [junit] at > >>> > >>> > >>> > org.apache.manifoldcf.agents.output.mongodboutput.tests.APISanityHSQLDBIT.removeTestArea(APISanityHSQLDBIT.java:109) > >>> > >>> [junit] at > >>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > >>> Method) > >>> [junit] at > >>> > >>> > >>> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > >>> > >>> [junit] at > >>> > >>> > >>> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >>> > >>> ``` > >>> > >>> And no more success for the ITs tests of Solr connector... > >>> > >>> Guylaine > >>> Le 25/10/2023 à 12:52, Karl Wright a écrit : > >>> > >>> I was able to reproduce the problem last night. I believe the cause > may > >>> well be that we've moved too many dependencies to the framework level. > >>> Specifically, I think perhaps only zookeeper and netty should run > there, > >>> but the Scala library probably needs to run at the same classloader > level > >>> as the other Scala jars, so it should be moved back into > connectors/kafka > >>> and removed from framework/build.xml and from build.xml and from > >>> connector-build.xml. > >>> > >>> Sadly I'm completely snowed under until the weekend so it will need to > >>> > >>> wait > >>> > >>> until then unless someone else wants to try this. > >>> > >>> Karl > >>> > >>> > >>> On Tue, Oct 24, 2023 at 9:12 AM Karl Wright<daddy...@gmail.com> < > daddy...@gmail.com> < > >>> > >>> daddy...@gmail.com> wrote: > >>> > >>> Try doing svn update and deleting your test-materials directory > contents. > >>> Then ant download-dependencies. You don't get a link error after that > >>> > >>> when > >>> > >>> you do run-IT-HSQLDB . > >>> > >>> Karl > >>> > >>> On Tue, Oct 24, 2023 at 9:10 AM Guylaine BASSETTE < > >>> > >>> guylaine.basse...@francelabs.com> wrote: > >>> > >>> Hi Karl and Mingchun, > >>> > >>> Thanks again ! > >>> > >>> My bad for the Zookeeper dependencies. Actually I've made a mistake > >>> using in my IDE dependencies analyzer. > >>> > >>> Regarding Kafka tests, a light search brings me to the > >>> "spark-streaming-kafka" dependency that might be missing... > >>> > >>> At the mean time, I continue my effort on Solr connector tests. > >>> > >>> > >>> For any use, here is the errors I get: > >>> > >>> ``` > >>> > >>> [junit] [main] INFO > org.eclipse.jetty.server.handler.ContextHandler - > >>> Stopped o.e.j.w.WebAppContext@7c28c1{ManifoldCF General API > >>> > >>> > >>> > >>> > Webapp,/mcf-api-service,null,STOPPED}{/home/guylaine/IdeaProjects/mon-manifoldcf/dist/web/war/mcf-api-service.war} > >>> > >>> [junit] [main] INFO > org.eclipse.jetty.server.handler.ContextHandler > >>> - Stopped o.e.j.w.WebAppContext@588ffeb{ManifoldCF Authorities API > >>> > >>> > >>> > >>> > Webapp,/mcf-authority-service,null,STOPPED}{/home/guylaine/IdeaProjects/mon-manifoldcf/dist/web/war/mcf-authority-service.war} > >>> > >>> [junit] [main] INFO > org.eclipse.jetty.server.handler.ContextHandler > >>> - Stopped o.e.j.w.WebAppContext@71a3a190{ManifoldCF Crawler > >>> > >>> > >>> > >>> > Interface,/mcf-crawler-ui,null,STOPPED}{/home/guylaine/IdeaProjects/mon-manifoldcf/dist/web/war/mcf-crawler-ui.war} > >>> > >>> [junit] ------------- ---------------- --------------- > >>> [junit] Testcase: > >>> > sanityCheck(org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT): > >>> Caused an ERROR > >>> [junit] 'scala.collection.immutable.ArraySeq > >>> scala.runtime.ScalaRunTime$.wrapRefArray(java.lang.Object[])' > >>> [junit] java.lang.NoSuchMethodError: > >>> 'scala.collection.immutable.ArraySeq > >>> scala.runtime.ScalaRunTime$.wrapRefArray(java.lang.Object[])' > >>> [junit] at > >>> kafka.server.KafkaConfig$.<clinit>(KafkaConfig.scala:338) > >>> [junit] at > >>> kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1603) > >>> [junit] at > >>> > >>> > >>> > >>> > org.apache.manifoldcf.agents.output.kafka.KafkaLocal.<init>(KafkaLocal.java:31) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > org.apache.manifoldcf.agents.output.kafka.BaseITHSQLDB.setupKafka(BaseITHSQLDB.java:86) > >>> > >>> [junit] at > >>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > >>> Method) > >>> [junit] at > >>> > >>> > >>> > >>> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >>> > >>> [junit] > >>> [junit] > >>> [junit] Testcase: > >>> > sanityCheck(org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT): > >>> Caused an ERROR > >>> [junit] null > >>> [junit] java.lang.NullPointerException > >>> [junit] at > >>> > >>> > >>> > >>> > org.apache.manifoldcf.agents.output.kafka.BaseITHSQLDB.cleanUpKafka(BaseITHSQLDB.java:92) > >>> > >>> [junit] at > >>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > >>> Method) > >>> [junit] at > >>> > >>> > >>> > >>> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >>> > >>> ``` > >>> > >>> Le 24/10/2023 à 12:30, Karl Wright a écrit : > >>> > >>> I missed a place - connector-build.xml. Updated now. > >>> > >>> Now we don't get a link exception, but neither does the kafka test > work. > >>> It seems to be unable to start zookeeper even though all the > >>> > >>> dependencies > >>> > >>> are now there. Will need to look at this after work. > >>> > >>> Karl > >>> > >>> > >>> On Mon, Oct 23, 2023 at 11:32 PM Mingchun Zhao< > >>> mingchun.zha...@gmail.com> > >>> > >>> wrote: > >>> > >>> > >>> Thanks. I've tried `ant test` with the latest trunk. As a result, the > >>> > >>> kafka > >>> > >>> test failed as below. > >>> ``` > >>> [junit] Testcase: > >>> > >>> > >>> > sanityCheck(org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT): > >>> > >>> Caused an ERROR > >>> [junit] io/netty/handler/ssl/SslContext > >>> [junit] java.lang.NoClassDefFoundError: > >>> > >>> io/netty/handler/ssl/SslContext > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.zookeeper.common.ZKConfig.handleBackwardCompatibility(ZKConfig.java:106) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.zookeeper.client.ZKClientConfig.handleBackwardCompatibility(ZKClientConfig.java:96) > >>> > >>> [junit] at > >>> org.apache.zookeeper.common.ZKConfig.init(ZKConfig.java:92) > >>> [junit] at > >>> org.apache.zookeeper.common.ZKConfig.<init>(ZKConfig.java:61) > >>> [junit] at > >>> > >>> > >>> > org.apache.zookeeper.client.ZKClientConfig.<init>(ZKClientConfig.java:69) > >>> > >>> [junit] at > >>> > >>> kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1620) > >>> > >>> [junit] at > >>> > >>> kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1603) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.agents.output.kafka.KafkaLocal.<init>(KafkaLocal.java:31) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.agents.output.kafka.BaseITHSQLDB.setupKafka(BaseITHSQLDB.java:86) > >>> > >>> [junit] at > >>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > >>> Method) > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >>> > >>> [junit] Caused by: java.lang.ClassNotFoundException: > >>> io.netty.handler.ssl.SslContext > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) > >>> > >>> [junit] at > >>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) > >>> [junit] > >>> [junit] > >>> [junit] Testcase: > >>> > >>> > >>> > sanityCheck(org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT): > >>> > >>> Caused an ERROR > >>> [junit] null > >>> [junit] java.lang.NullPointerException > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.agents.output.kafka.BaseITHSQLDB.cleanUpKafka(BaseITHSQLDB.java:92) > >>> > >>> [junit] at > >>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > >>> Method) > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >>> > >>> [junit] > >>> [junit] > >>> > >>> BUILD FAILED > >>> /Users/zhaomingchun/ManifoldCF/manifoldcf/build.xml:517: The following > >>> error occurred while executing this line: > >>> /Users/zhaomingchun/ManifoldCF/manifoldcf/build.xml:471: The following > >>> error occurred while executing this line: > >>> > >>> > >>> > /Users/zhaomingchun/ManifoldCF/manifoldcf/dist/connector-build.xml:1102: > >>> > >>> Test org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT failed > >>> ``` > >>> > >>> 2023年10月24日(火) 11:00 Karl Wright<daddy...@gmail.com> < > daddy...@gmail.com> <daddy...@gmail.com > >>> : > >>> > >>> > >>> Okay, I updated zookeeper properly in build.xml and > >>> > >>> framework/build.xml, > >>> > >>> with the two new dependencies, and the zookeeper tests pass. I > >>> > >>> haven't > >>> > >>> tried the kafka or solr tests yet. > >>> > >>> Karl > >>> > >>> > >>> On Mon, Oct 23, 2023 at 9:29 PM Karl Wright<daddy...@gmail.com> < > daddy...@gmail.com> < > >>> > >>> daddy...@gmail.com> > >>> > >>> wrote: > >>> > >>> Unless I know what kafka is using zookeeper for, this would seem > >>> > >>> risky > >>> > >>> to > >>> > >>> me. Zookeeper is meant to coordinate processes; it may not work for > >>> > >>> one > >>> > >>> process to be using different versions of zookeeper than the others. > >>> > >>> It looks like the original change to kafka you reverted had the > >>> > >>> proper > >>> > >>> dependencies but they absolutely needed to be included in the right > >>> classpaths and they weren't - they were only included in the kafka > >>> > >>> tests. > >>> > >>> I will look at this perhaps at the latest this weekend, but I won't > >>> > >>> commit > >>> > >>> this patch. > >>> > >>> > >>> Karl > >>> > >>> > >>> On Mon, Oct 23, 2023 at 5:14 PM Mingchun Zhao < > >>> mingchun.zha...@gmail.com > >>> > >>> wrote: > >>> > >>> > >>> I reverted zookeeper version to 3.8.0 to avoid linkage error on the > >>> multiThreadZooKeeperLockTest: > >>> [junit] Caused by: java.lang.ClassNotFoundException: > >>> io.netty.handler.ssl.SslContext > >>> > >>> I've prepared a PR here:https://github.com/apache/manifoldcf/pull/156 > >>> > >>> Just a heads up, `ant test` still hangs on the Solr Output connector > >>> > >>> test: > >>> > >>> ``` > >>> run-IT-HSQLDB: > >>> [junit] Testsuite: > >>> org.apache.manifoldcf.agents.output.solr.tests.SolrCrawlHSQLDBIT > >>> [junit] Configuration file successfully read > >>> [junit] [main] INFO org.eclipse.jetty.util.log - Logging > >>> > >>> initialized > >>> > >>> @7027ms to org.eclipse.jetty.util.log.Slf4jLog > >>> [junit] [main] INFO org.eclipse.jetty.server.Server - > >>> jetty-9.4.48.v20220622; built: 2022-06-21T20:42:25.880Z; git: > >>> 6b67c5719d1f4371b33655ff2d047d24e171e49a; jvm 11.0.11+9 > >>> [junit] [main] INFO org.eclipse.jetty.server.session - > >>> DefaultSessionIdManager workerName=node0 > >>> [junit] [main] INFO org.eclipse.jetty.server.session - No > >>> SessionScavenger set, using defaults > >>> [junit] [main] INFO org.eclipse.jetty.server.session - node0 > >>> Scavenging > >>> every 660000ms > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.handler.ContextHandler > >>> > >>> - > >>> > >>> Started o.e.j.w.WebAppContext@1517f633{ManifoldCF Crawler > >>> > >>> > >>> > >>> > >>> > >>> > Interface,/mcf-crawler-ui,file:///private/var/folders/zh/mx4q_qh93cv6jtp13ht8b1fr0000gn/T/jetty-0_0_0_0-8346-mcf-crawler-ui_war-_mcf-crawler-ui-any-7554899724821045499/webapp/,AVAILABLE}{/Users/zhaomingchun/ManifoldCF/manifoldcf/dist/web/war/mcf-crawler-ui.war} > >>> > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.handler.ContextHandler > >>> > >>> - > >>> > >>> Started o.e.j.w.WebAppContext@4fe01803{ManifoldCF Authorities API > >>> > >>> > >>> > >>> > >>> > >>> > Webapp,/mcf-authority-service,file:///private/var/folders/zh/mx4q_qh93cv6jtp13ht8b1fr0000gn/T/jetty-0_0_0_0-8346-mcf-authority-service_war-_mcf-authority-service-any-7701836901953162228/webapp/,AVAILABLE}{/Users/zhaomingchun/ManifoldCF/manifoldcf/dist/web/war/mcf-authority-service.war} > >>> > >>> [junit] Creating mock service > >>> [junit] Mock service created > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.handler.ContextHandler > >>> > >>> - > >>> > >>> Started o.e.j.w.WebAppContext@13d186db{ManifoldCF General API > >>> > >>> > >>> > >>> > >>> > >>> > Webapp,/mcf-api-service,file:///private/var/folders/zh/mx4q_qh93cv6jtp13ht8b1fr0000gn/T/jetty-0_0_0_0-8346-mcf-api-service_war-_mcf-api-service-any-2609388202403972652/webapp/,AVAILABLE}{/Users/zhaomingchun/ManifoldCF/manifoldcf/dist/web/war/mcf-api-service.war} > >>> > >>> [junit] [main] INFO org.eclipse.jetty.server.AbstractConnector > >>> > >>> - > >>> > >>> Started ServerConnector@3bd55d8{HTTP/1.1, (http/1.1)}{0.0.0.0:8346} > >>> [junit] [main] INFO org.eclipse.jetty.server.Server - Started > >>> > >>> @9054ms > >>> > >>> [junit] [main] INFO org.eclipse.jetty.server.Server - > >>> jetty-9.4.48.v20220622; built: 2022-06-21T20:42:25.880Z; git: > >>> 6b67c5719d1f4371b33655ff2d047d24e171e49a; jvm 11.0.11+9 > >>> [junit] [main] INFO org.eclipse.jetty.server.session - > >>> DefaultSessionIdManager workerName=node0 > >>> [junit] [main] INFO org.eclipse.jetty.server.session - No > >>> SessionScavenger set, using defaults > >>> [junit] [main] INFO org.eclipse.jetty.server.session - node0 > >>> Scavenging > >>> every 600000ms > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.handler.ContextHandler > >>> > >>> - > >>> > >>> Started o.e.j.s.ServletContextHandler@6f4ade6e > >>> > >>> {/solr,null,AVAILABLE} > >>> > >>> [junit] [main] INFO org.eclipse.jetty.server.AbstractConnector > >>> > >>> - > >>> > >>> Started ServerConnector@30e6a763{HTTP/1.1, (http/1.1)}{0.0.0.0:8188 > >>> > >>> } > >>> > >>> [junit] [main] INFO org.eclipse.jetty.server.Server - Started > >>> > >>> @9064ms > >>> > >>> [junit] [main] INFO org.eclipse.jetty.server.AbstractConnector > >>> > >>> - > >>> > >>> Stopped ServerConnector@30e6a763{HTTP/1.1, (http/1.1)}{0.0.0.0:8188 > >>> > >>> } > >>> > >>> [junit] [main] INFO org.eclipse.jetty.server.session - node0 > >>> > >>> Stopped > >>> > >>> scavenging > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.handler.ContextHandler > >>> > >>> - > >>> > >>> Stopped o.e.j.s.ServletContextHandler@6f4ade6e{/solr,null,STOPPED} > >>> ``` > >>> > >>> 2023年10月24日(火) 1:40 Karl Wright<daddy...@gmail.com> < > daddy...@gmail.com> <daddy...@gmail.com > >>> : > >>> > >>> > >>> The dependencies would be in the zookeeper pom. Maven would follow > >>> > >>> them > >>> > >>> automatically which is why it is insufficient to assume that if > >>> > >>> maven > >>> > >>> works > >>> > >>> so will ant. > >>> > >>> You can use mvn dependency:tree to find what Maven is actually > >>> > >>> pulling > >>> > >>> in. > >>> > >>> Karl > >>> > >>> > >>> On Mon, Oct 23, 2023 at 11:04 AM Guylaine BASSETTE < > >>> > >>> guylaine.basse...@francelabs.com> wrote: > >>> > >>> I launch these tests with a Maven build and everything is OK. Yet > >>> > >>> no > >>> > >>> netty dependencies are requiered... I don't understand were this > >>> SslContext is called... > >>> > >>> Le 23/10/2023 à 16:25, Karl Wright a écrit : > >>> > >>> Yes, that is indicating that zookeeper is looking for a specific > >>> > >>> netty > >>> > >>> class that it isn't finding. That is why I think there is now a > >>> > >>> zookeeper > >>> > >>> dependency we aren't including in the classpaths that include > >>> > >>> zookeeper. > >>> > >>> Karl > >>> > >>> > >>> On Mon, Oct 23, 2023 at 10:23 AM Mingchun Zhao< > >>> mingchun.zha...@gmail.com > >>> > >>> wrote: > >>> > >>> > >>> Karl, Thanks! > >>> I think I reproduced that multiThreadZooKeeperLockTest error > >>> > >>> when I > >>> > >>> running `ant test`, will look into this. > >>> ``` > >>> [junit] ------------- ---------------- --------------- > >>> [junit] Testcase: > >>> > >>> > >>> > >>> > >>> > >>> > multiThreadZooKeeperLockTest(org.apache.manifoldcf.core.lockmanager.TestZooKeeperLocks): > >>> > >>> Caused an ERROR > >>> [junit] io/netty/handler/ssl/SslContext > >>> [junit] java.lang.NoClassDefFoundError: > >>> > >>> io/netty/handler/ssl/SslContext > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.zookeeper.common.ZKConfig.handleBackwardCompatibility(ZKConfig.java:106) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.zookeeper.client.ZKClientConfig.handleBackwardCompatibility(ZKClientConfig.java:96) > >>> > >>> [junit] at > >>> org.apache.zookeeper.common.ZKConfig.init(ZKConfig.java:92) > >>> [junit] at > >>> org.apache.zookeeper.common.ZKConfig.<init>(ZKConfig.java:61) > >>> [junit] at > >>> > >>> > >>> > org.apache.zookeeper.client.ZKClientConfig.<init>(ZKClientConfig.java:69) > >>> > >>> [junit] at > >>> org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:643) > >>> [junit] at > >>> org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:567) > >>> [junit] at > >>> org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:734) > >>> [junit] at > >>> org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:448) > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.createSession(ZooKeeperConnection.java:74) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.<init>(ZooKeeperConnection.java:66) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.core.lockmanager.ZooKeeperConnectionPool.grab(ZooKeeperConnectionPool.java:48) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.core.lockmanager.ZooKeeperLockObject.obtainGlobalReadLock(ZooKeeperLockObject.java:190) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.core.lockmanager.LockObject.enterReadLock(LockObject.java:310) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.core.lockmanager.LockGate.enterReadLock(LockGate.java:271) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.core.lockmanager.TestZooKeeperLocks.enterReadLock(TestZooKeeperLocks.java:125) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.core.lockmanager.TestZooKeeperLocks$ReaderThread.run(TestZooKeeperLocks.java:204) > >>> > >>> [junit] Caused by: java.lang.ClassNotFoundException: > >>> io.netty.handler.ssl.SslContext > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) > >>> > >>> [junit] at > >>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) > >>> [junit] > >>> [junit] > >>> > >>> BUILD FAILED > >>> ``` > >>> > >>> 2023年10月23日(月) 23:02 Karl Wright<daddy...@gmail.com> < > daddy...@gmail.com> <daddy...@gmail.com > >>> : > >>> > >>> > >>> This is all you need to do: > >>> > >>> ant clean-core-deps > >>> ant make-core-deps > >>> ant clean > >>> ant test > >>> > >>> Karl > >>> > >>> > >>> On Mon, Oct 23, 2023 at 9:55 AM Mingchun Zhao < > >>> mingchun.zha...@gmail.com > >>> > >>> wrote: > >>> > >>> > >>> Hi Guylaine, Thanks! > >>> > >>> > >>> Thanks for all your sharing, it's very helpful! I'll > >>> > >>> continue... > >>> > >>> I'll look into it some more too. If I have any other > >>> > >>> information > >>> > >>> I'll > >>> > >>> share > >>> > >>> it with you. > >>> > >>> 2023年10月23日(月) 22:49 Guylaine BASSETTE < > >>> guylaine.basse...@francelabs.com > >>> > >>> : > >>> > >>> > >>> Thanks for all your sharing, it's very helpful! I'll > >>> > >>> continue... > >>> > >>> Sorry, French and English mixed up! > >>> > >>> Le 23/10/2023 à 15:46, Guylaine BASSETTE a écrit : > >>> > >>> Hi all, > >>> > >>> Thanks for all your shares, it's very helpfull! Merci pour > >>> > >>> tous > >>> > >>> vos > >>> > >>> partages, c'est très utile ! Je poursuis... > >>> > >>> Le 23/10/2023 à 15:31, Karl Wright a écrit : > >>> > >>> I can't give advice on the test; this is something > >>> > >>> FranceLabs > >>> > >>> should > >>> > >>> look > >>> at. > >>> However, nothing of what you are doing will affect the > >>> > >>> zookeeper > >>> > >>> tests in > >>> framework. That's a totally different issue. > >>> > >>> Karl > >>> > >>> > >>> On Mon, Oct 23, 2023 at 9:20 AM Mingchun Zhao< > >>> mingchun.zha...@gmail.com > >>> > >>> wrote: > >>> > >>> > >>> Hi Karl and Guylaine, > >>> > >>> > >>> I hope and think it's just a problem specific to the > >>> > >>> test. > >>> > >>> Missing > >>> > >>> updates or incompatible dependencies... > >>> > >>> Allow me to share with you what I'm working on. I've > >>> > >>> tried > >>> > >>> to > >>> > >>> support > >>> > >>> http2C within the Solr output connector junit test, but > >>> > >>> got > >>> > >>> another > >>> > >>> unhandled solr exception when I ran `ant run-IT-HSQLDB`. > >>> > >>> - source code change > >>> ``` > >>> diff --git > >>> > >>> > >>> > >>> > >>> > >>> > a/connectors/solr/connector/src/test/java/org/apache/manifoldcf/agents/output/solr/tests > >>> > >>> /MockSolrService.java > >>> > >>> > >>> > >>> > >>> > >>> > b/connectors/solr/connector/src/test/java/org/apache/manifoldcf/agents/output > >>> > >>> /solr/tests/MockSolrService.java > >>> index 237ade09c..3fb558f52 100644 > >>> --- > >>> > >>> > >>> > >>> > >>> > >>> > a/connectors/solr/connector/src/test/java/org/apache/manifoldcf/agents/output/solr/tests/MockSo > >>> > >>> lrService.java > >>> +++ > >>> > >>> > >>> > >>> > >>> > >>> > b/connectors/solr/connector/src/test/java/org/apache/manifoldcf/agents/output/solr/tests/MockSo > >>> > >>> lrService.java > >>> @@ -18,7 +18,10 @@ > >>> */ > >>> package > >>> > >>> org.apache.manifoldcf.agents.output.solr.tests; > >>> > >>> +import > >>> > >>> org.eclipse.jetty.http2.server.HTTP2CServerConnectionFactory; > >>> > >>> import org.eclipse.jetty.servlet.ServletHolder; > >>> +import org.eclipse.jetty.server.HttpConfiguration; > >>> +import org.eclipse.jetty.server.HttpConnectionFactory; > >>> import org.eclipse.jetty.server.Server; > >>> import org.eclipse.jetty.server.ServerConnector; > >>> import > >>> > >>> org.eclipse.jetty.servlet.ServletContextHandler; > >>> > >>> @@ -40,7 +43,10 @@ public class MockSolrService > >>> public MockSolrService() > >>> { > >>> server = new Server(new QueuedThreadPool(35)); > >>> - ServerConnector connector = new > >>> > >>> ServerConnector(server); > >>> > >>> + HttpConfiguration config = new HttpConfiguration(); > >>> + HttpConnectionFactory http1 = new > >>> > >>> HttpConnectionFactory(config); > >>> > >>> + HTTP2CServerConnectionFactory http2c = new > >>> HTTP2CServerConnectionFactory(config); > >>> + ServerConnector connector = new > >>> > >>> ServerConnector(server, > >>> > >>> http1, > >>> > >>> http2c); > >>> connector.setPort(8188); > >>> server.addConnector(connector); > >>> servlet = new SolrServlet(); > >>> @@ -111,6 +117,7 @@ public class MockSolrService > >>> res.getWriter().printf(Locale.ROOT, "<solr>\n"); > >>> res.getWriter().printf(Locale.ROOT, > >>> > >>> "</solr>\n"); > >>> > >>> res.getWriter().flush(); > >>> } > >>> ``` > >>> > >>> - run Junit test > >>> ``` > >>> ~manifoldcf/connectors/solr% ant run-IT-HSQLDB > >>> ``` > >>> > >>> - confirm test-HSQLDB-output/manifoldcf.log > >>> ``` > >>> ERROR 2023-10-23T22:10:29,902 (Worker thread '24') - > >>> > >>> Exception > >>> > >>> tossed: > >>> > >>> Unhandled Solr exception during indexinghttp://test70.txt > >>> > >>> (200): > >>> > >>> Error > >>> > >>> from server athttp://localhost:8188/solr: Expected mime > >>> > >>> type > >>> > >>> application/octet-stream but got application/xml. > >>> > >>> <result> > >>> > >>> <doc name="something"/> > >>> </result> > >>> > >>> > >>> > >>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: > >>> > >>> Unhandled > >>> > >>> Solr > >>> exception during indexinghttp://test70.txt (200): Error > >>> > >>> from > >>> > >>> server > >>> > >>> at > >>> http://localhost:8188/solr: Expected mime type > >>> > >>> application/octet-stream > >>> > >>> but > >>> got application/xml. <result> > >>> <doc name="something"/> > >>> </result> > >>> > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.agents.output.solr.HttpPoster.handleSolrException(HttpPoster.java:389) > >>> > >>> ~[classes/:?] > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.agents.output.solr.HttpPoster.indexPost(HttpPoster.java:544) > >>> > >>> ~[classes/:?] > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.agents.output.solr.SolrConnector.addOrReplaceDocumentWithException(SolrConnector.java:522) > >>> > >>> ~[classes/:?] > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3214) > >>> > >>> ~[mcf-agents.jar:?] > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$OutputAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3395) > >>> > >>> ~[mcf-agents.jar:?] > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3065) > >>> > >>> ~[mcf-agents.jar:?] > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2696) > >>> > >>> ~[mcf-agents.jar:?] > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:750) > >>> > >>> ~[mcf-agents.jar:?] > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1585) > >>> > >>> ~[mcf-pull-agent.jar:?] > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1550) > >>> > >>> ~[mcf-pull-agent.jar:?] > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.crawler.tests.TestingRepositoryConnector.processDocuments(TestingRepositoryConnector.java:84) > >>> > >>> ~[mcf-pull-agent-tests.jar:?] > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:402) > >>> > >>> ~[mcf-pull-agent.jar:?] > >>> ERROR 2023-10-23T22:10:29,902 (Worker thread '10') - > >>> > >>> Exception > >>> > >>> tossed: > >>> > >>> Unhandled Solr exception during indexinghttp://test72.txt > >>> > >>> (200): > >>> > >>> Error > >>> > >>> from server athttp://localhost:8188/solr: Expected mime > >>> > >>> type > >>> > >>> application/octet-stream but got application/xml. > >>> > >>> <result> > >>> > >>> <doc name="something"/> > >>> </result> > >>> ``` > >>> > >>> Could you give me some advice? > >>> > >>> 2023年10月23日(月) 22:01 Mingchun Zhao< > >>> mingchun.zha...@gmail.com > >>> > >>> : > >>> > >>> Then, wherever zookeeper is mentioned in > >>> > >>> framework/build.xml, > >>> > >>> a > >>> > >>> reference to those dependencies must also be included. > >>> > >>> It looks like zookeeper*.jar was already included in > >>> connector-test-classpath within kafka/build.xml. > >>> ``` > >>> <path id="connector-test-classpath"> > >>> <path > >>> > >>> refid="mcf-connector-build.connector-test-classpath"/> > >>> > >>> <fileset dir="../../lib"> > >>> <include name="zookeeper*.jar"/> > >>> <include name="kafka-clients*.jar"/> > >>> <include name="lz4*.jar"/> > >>> <include name="snappy-java*.jar"/> > >>> <include name="metrics-core*.jar"/> > >>> </fileset> > >>> ``` > >>> > >>> > >>> 2023年10月23日(月) 21:50 Karl Wright<daddy...@gmail.com> < > daddy...@gmail.com> <daddy...@gmail.com > >>> : > >>> > >>> > >>> Hi, > >>> > >>> That just downloads zookeeper. But apparently the > >>> > >>> zookeeper > >>> > >>> version > >>> > >>> required by Kafka now has dependencies of its own. > >>> > >>> Otherwise > >>> > >>> the > >>> > >>> zookeeper > >>> tests wouldn't fail with linkage errors. > >>> > >>> The dependencies need to be identified and added in > >>> > >>> several > >>> > >>> places. The > >>> first place is to the download-zookeeper part of the > >>> > >>> root > >>> > >>> build.xml > >>> > >>> script. Then, wherever zookeeper is mentioned in > >>> framework/build.xml, a > >>> reference to those dependencies must also be included. > >>> > >>> Karl > >>> > >>> > >>> On Mon, Oct 23, 2023 at 8:32 AM Mingchun Zhao < > >>> mingchun.zha...@gmail.com> > >>> > >>> wrote: > >>> > >>> > >>> Hi Karl, > >>> > >>> > >>> Mingchun, did you add the jar(s) that the new > >>> > >>> zookeeper > >>> > >>> needs > >>> > >>> to > >>> > >>> the > >>> > >>> build.xml download section? > >>> > >>> Are the following settings correct? Or do you have an > >>> > >>> old > >>> > >>> version > >>> > >>> of > >>> > >>> zookeeper-*.jar left in your environment? > >>> > >>> ```build.xml > >>> <target name="download-zookeeper"> > >>> <mkdir dir="lib"/> > >>> <antcall target="download-via-maven"> > >>> <param name="target" value="lib"/> > >>> <param name="project-path" > >>> value="org/apache/zookeeper"/> > >>> <param name="artifact-version" > >>> > >>> value="${zookeeper.version}"/> > >>> > >>> <param name="artifact-name" > >>> > >>> value="zookeeper"/> > >>> > >>> <param name="artifact-type" > >>> > >>> value="jar"/> > >>> > >>> </antcall> > >>> ... ... > >>> ``` > >>> > >>> Regards, > >>> Mingchun > >>> > >>> > >>> 2023年10月23日(月) 21:19 Karl Wright<daddy...@gmail.com> < > daddy...@gmail.com> <daddy...@gmail.com > >>> : > >>> > >>> > >>> Well, that was interesting. > >>> > >>> Didn't get very far, because the dependency updates > >>> > >>> committed > >>> > >>> broke > >>> > >>> Zookeeper again: > >>> > >>> [junit] Testcase: > >>> > >>> > >>> > >>> > >>> > >>> > multiThreadZooKeeperLockTest(org.apache.manifoldcf.core.lockmanager.TestZooKeeperLocks): > >>> > >>> Caused an ERROR > >>> [junit] io/netty/handler/ssl/SslContext > >>> [junit] java.lang.NoClassDefFoundError: > >>> > >>> io/netty/handler/ssl/SslContext > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.zookeeper.common.ZKConfig.handleBackwardCompatibility(ZKConfig.java:106) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.zookeeper.client.ZKClientConfig.handleBackwardCompatibility(ZKClientConfig.java:96) > >>> > >>> [junit] at > >>> > >>> > >>> org.apache.zookeeper.common.ZKConfig.init(ZKConfig.java:92) > >>> > >>> [junit] at > >>> > >>> > >>> org.apache.zookeeper.common.ZKConfig.<init>(ZKConfig.java:61) > >>> > >>> [junit] at > >>> > >>> > >>> > org.apache.zookeeper.client.ZKClientConfig.<init>(ZKClientConfig.java:69) > >>> > >>> [junit] at > >>> > >>> > >>> org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:643) > >>> > >>> [junit] at > >>> > >>> > >>> org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:567) > >>> > >>> [junit] at > >>> > >>> > >>> org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:734) > >>> > >>> [junit] at > >>> > >>> > >>> org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:448) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.createSession(ZooKeeperConnection.java:74) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.<init>(ZooKeeperConnection.java:66) > >>> > >>> It looks like the even newer Zookeeper version has a > >>> > >>> newer > >>> > >>> dependency > >>> > >>> that > >>> > >>> isn't being included in the basic classpath, but > >>> > >>> should > >>> > >>> be. > >>> > >>> Mingchun, > >>> > >>> did > >>> > >>> you add the jar(s) that the new zookeeper needs to > >>> > >>> the > >>> > >>> build.xml > >>> > >>> download > >>> > >>> section? If so, can you remind me what they were? > >>> > >>> Karl > >>> > >>> > >>> On Mon, Oct 23, 2023 at 8:11 AM Karl Wright< > >>> daddy...@gmail.com > >>> > >>> wrote: > >>> > >>> I begin to suspect that the problem may be human > >>> > >>> error. > >>> > >>> If you don't do "ant clean-core-deps; ant > >>> > >>> make-core-deps", > >>> > >>> but > >>> > >>> instead > >>> > >>> just use "ant make-core-deps", you could have > >>> > >>> incompatible > >>> > >>> versions > >>> > >>> of > >>> > >>> several libraries in your classpath for the tests. > >>> > >>> I'll > >>> > >>> try > >>> > >>> today > >>> > >>> to > >>> > >>> verify whether that might be happening by trying the > >>> > >>> tests > >>> > >>> locally > >>> > >>> myself. > >>> > >>> Karl > >>> > >>> > >>> On Mon, Oct 23, 2023 at 7:57 AM Guylaine BASSETTE < > >>> > >>> guylaine.basse...@francelabs.com> wrote: > >>> > >>> Hi Karl and Mingchun, > >>> > >>> Thanks for your work on the last few issues. I join > >>> > >>> you on > >>> > >>> this > >>> > >>> Solr > >>> > >>> testing problem. > >>> > >>> That said, we've tested this new connector in our > >>> > >>> application, > >>> > >>> with a > >>> > >>> FileShare job and everything was OK. > >>> > >>> I hope and think it's just a problem specific to > >>> > >>> the > >>> > >>> test. > >>> > >>> Missing > >>> > >>> updates or incompatible dependencies... > >>> > >>> Le 20/10/2023 à 02:58, Mingchun Zhao a écrit : > >>> > >>> Hi Karl, Thanks! > >>> > >>> > >>> so I wonder if, once again, there's a problem > >>> > >>> with > >>> > >>> dependencies > >>> > >>> for > >>> > >>> the > >>> > >>> version of Solr they chose. > >>> > >>> I'll take a look at this issue. > >>> > >>> 2023年10月20日(金) 9:50 Karl Wright< > >>> daddy...@gmail.com > >>> > >>> : > >>> > >>> This connector FranceLabs updated. The problem > >>> > >>> seems to > >>> > >>> occur > >>> > >>> at a > >>> > >>> basic > >>> > >>> level during http2 communication, so I wonder if, > >>> > >>> once > >>> > >>> again, > >>> > >>> there's a > >>> > >>> problem with dependencies for the version of Solr > >>> > >>> they > >>> > >>> chose. > >>> > >>> Karl > >>> > >>> > >>> On Thu, Oct 19, 2023 at 8:32 PM Mingchun Zhao< > >>> mingchun.zha...@gmail.com> > >>> > >>> wrote: > >>> > >>> > >>> About the test "SolrCrawlHSQLDBIT" failure, it > >>> > >>> seems > >>> > >>> that > >>> > >>> "IO > >>> > >>> exception > >>> > >>> during indexinghttp://test58.txt: > >>> > >>> frame_size_error/invalid_frame_length" > >>> > >>> error is occurring on the ManifoldCF side. > >>> > >>> - command: > >>> ``` > >>> manifoldcf/connectors/solr% ant run-IT-HSQLDB > >>> > >>> run-IT-HSQLDB: > >>> [junit] Testsuite: > >>> > >>> > >>> org.apache.manifoldcf.agents.output.solr.tests.SolrCrawlHSQLDBIT > >>> > >>> ... ... > >>> ``` > >>> > >>> - I checked > >>> > >>> "connectors/solr/test-HSQLDB-output/manifoldcf.log": > >>> > >>> ``` > >>> WARN 2023-10-20T09:14:56,635 (Worker thread > >>> > >>> '18') - > >>> > >>> IO > >>> > >>> exception > >>> > >>> during > >>> > >>> indexinghttp://test58.txt: > >>> > >>> frame_size_error/invalid_frame_length > >>> > >>> java.io.IOException: > >>> > >>> frame_size_error/invalid_frame_length > >>> > >>> at > >>> > >>> org.eclipse.jetty.http2.HTTP2Session.toFailure(HTTP2Session.java:566) > >>> > >>> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> org.eclipse.jetty.http2.HTTP2Session.access$2700(HTTP2Session.java:80) > >>> > >>> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.eclipse.jetty.http2.HTTP2Session$StreamsState.onSessionFailure(HTTP2Session.java:1857) > >>> > >>> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.eclipse.jetty.http2.HTTP2Session$StreamsState.access$400(HTTP2Session.java:1436) > >>> > >>> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.eclipse.jetty.http2.HTTP2Session.onSessionFailure(HTTP2Session.java:511) > >>> > >>> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.eclipse.jetty.http2.HTTP2Session.onConnectionFailure(HTTP2Session.java:506) > >>> > >>> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.eclipse.jetty.http2.parser.Parser$Listener$Wrapper.onConnectionFailure(Parser.java:414) > >>> > >>> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.eclipse.jetty.http2.HTTP2Connection$ParserListener.onConnectionFailure(HTTP2Connection.java:397) > >>> > >>> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.eclipse.jetty.http2.parser.BodyParser.notifyConnectionFailure(BodyParser.java:223) > >>> > >>> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.eclipse.jetty.http2.parser.BodyParser.connectionFailure(BodyParser.java:215) > >>> > >>> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > org.eclipse.jetty.http2.parser.Parser.connectionFailure(Parser.java:209) > >>> > >>> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> org.eclipse.jetty.http2.parser.Parser.parseHeader(Parser.java:155) > >>> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> org.eclipse.jetty.http2.parser.Parser.parse(Parser.java:121) > >>> > >>> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.eclipse.jetty.http2.HTTP2Connection$HTTP2Producer.produce(HTTP2Connection.java:261) > >>> > >>> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) > >>> > >>> ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) > >>> > >>> ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) > >>> > >>> ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) > >>> > >>> ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > >>> > org.eclipse.jetty.http2.HTTP2Connection.produce(HTTP2Connection.java:183) > >>> > >>> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.eclipse.jetty.http2.HTTP2Connection.onFillable(HTTP2Connection.java:138) > >>> > >>> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.eclipse.jetty.http2.HTTP2Connection$FillableCallback.succeeded(HTTP2Connection.java:361) > >>> > >>> ~[http2-common-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at org.eclipse.jetty.io > >>> > >>> .FillInterest.fillable(FillInterest.java:105) > >>> > >>> ~[jetty-io-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at org.eclipse.jetty.io > >>> > >>> .ChannelEndPoint$1.run(ChannelEndPoint.java:104) > >>> > >>> ~[jetty-io-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.eclipse.jetty.util.thread.Invocable.invokeNonBlocking(Invocable.java:69) > >>> > >>> ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.invokeTask(EatWhatYouKill.java:350) > >>> > >>> ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:305) > >>> > >>> ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) > >>> > >>> ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) > >>> > >>> ~[jetty-util-9.4.48.v20220622.jar:9.4.48.v20220622] > >>> > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:271) > >>> > >>> ~[solr-solrj-9.1.0.jar:9.1.0 > >>> > >>> aa4f3d98ab19c201e7f3c74cd14c99174148616d > >>> > >>> - > >>> > >>> ishan - 2022-11-11 13:00:47] > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > >>> > >>> ~[?:?] > >>> at > >>> > >>> > >>> > >>> > >>> > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > >>> > >>> ~[?:?] > >>> at java.lang.Thread.run(Thread.java:829) ~[?:?] > >>> ``` > >>> > >>> 2023年10月19日(木) 20:37 Mingchun > >>> Zhao<mingchun.zha...@gmail.com > >>> > >>> : > >>> > >>> I'm having trouble with the Solr test `ant > >>> > >>> run-IT-HSQLDB`. > >>> > >>> It's been over an hour since I ran the test and > >>> > >>> it > >>> > >>> still > >>> > >>> doesn't > >>> > >>> finish. > >>> > >>> It seems that the problem is probably due to > >>> > >>> the > >>> > >>> Manifold > >>> > >>> job > >>> > >>> not > >>> > >>> finishing. > >>> Do you have any ideas or advice? Thanks! > >>> > >>> ``` > >>> ~/ManifoldCF/manifoldcf/connectors/solr% ant > >>> > >>> run-IT-HSQLDB > >>> > >>> Buildfile: > >>> > >>> > >>> /Users/zhaomingchun/ManifoldCF/manifoldcf/connectors/solr/build.xml > >>> > >>> ... ... > >>> run-IT-HSQLDB: > >>> [junit] Testsuite: > >>> > >>> > >>> org.apache.manifoldcf.agents.output.solr.tests.SolrCrawlHSQLDBIT > >>> > >>> [junit] Configuration file successfully > >>> > >>> read > >>> > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.util.log > >>> > >>> - > >>> > >>> Logging > >>> > >>> initialized > >>> > >>> @7416ms to org.eclipse.jetty.util.log.Slf4jLog > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.Server - > >>> > >>> jetty-9.4.48.v20220622; built: > >>> > >>> 2022-06-21T20:42:25.880Z; > >>> > >>> git: > >>> > >>> 6b67c5719d1f4371b33655ff2d047d24e171e49a; jvm > >>> > >>> 11.0.11+9 > >>> > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.session > >>> > >>> - > >>> > >>> DefaultSessionIdManager workerName=node0 > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.session > >>> > >>> - > >>> > >>> No > >>> > >>> SessionScavenger set, using defaults > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.session > >>> > >>> - > >>> > >>> node0 > >>> > >>> Scavenging every 600000ms > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.handler.ContextHandler - > >>> > >>> Started o.e.j.w.WebAppContext@4bdc8b5d > >>> > >>> {ManifoldCF > >>> > >>> Crawler > >>> > >>> > >>> > >>> > Interface,/mcf-crawler-ui,file:///private/var/folders/zh/mx4q_qh93cv6jtp13ht8b1fr0000gn/T/jetty-0_0_0_0-8346-mcf-crawler-ui_war-_mcf-crawler-ui-any-17189990303852051874/webapp/,AVAILABLE}{/Users/zhaomingchun/ManifoldCF/manifoldcf/dist/web/war/mcf-crawler-ui.war} > >>> > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.handler.ContextHandler - > >>> > >>> Started o.e.j.w.WebAppContext@3bcd426c > >>> > >>> {ManifoldCF > >>> > >>> Authorities > >>> > >>> API > >>> > >>> > >>> > >>> > Webapp,/mcf-authority-service,file:///private/var/folders/zh/mx4q_qh93cv6jtp13ht8b1fr0000gn/T/jetty-0_0_0_0-8346-mcf-authority-service_war-_mcf-authority-service-any-1541416273384984853/webapp/,AVAILABLE}{/Users/zhaomingchun/ManifoldCF/manifoldcf/dist/web/war/mcf-authority-service.war} > >>> > >>> [junit] Creating mock service > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.handler.ContextHandler - > >>> > >>> Started o.e.j.w.WebAppContext@5f14a673 > >>> > >>> {ManifoldCF > >>> > >>> General > >>> > >>> API > >>> > >>> > >>> > >>> > >>> > Webapp,/mcf-api-service,file:///private/var/folders/zh/mx4q_qh93cv6jtp13ht8b1fr0000gn/T/jetty-0_0_0_0-8346-mcf-api-service_war-_mcf-api-service-any-12421572948842353797/webapp/,AVAILABLE}{/Users/zhaomingchun/ManifoldCF/manifoldcf/dist/web/war/mcf-api-service.war} > >>> > >>> [junit] Mock service created > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.AbstractConnector - > >>> > >>> Started ServerConnector@70325d20{HTTP/1.1, > >>> > >>> (http/1.1)}{ > >>> 0.0.0.0:8346} > >>> > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.Server - > >>> > >>> Started > >>> > >>> @9616ms > >>> > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.Server - > >>> > >>> jetty-9.4.48.v20220622; built: > >>> > >>> 2022-06-21T20:42:25.880Z; > >>> > >>> git: > >>> > >>> 6b67c5719d1f4371b33655ff2d047d24e171e49a; jvm > >>> > >>> 11.0.11+9 > >>> > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.session > >>> > >>> - > >>> > >>> DefaultSessionIdManager workerName=node0 > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.session > >>> > >>> - > >>> > >>> No > >>> > >>> SessionScavenger set, using defaults > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.session > >>> > >>> - > >>> > >>> node0 > >>> > >>> Scavenging every 600000ms > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.handler.ContextHandler - > >>> > >>> Started o.e.j.s.ServletContextHandler@7808f638 > >>> > >>> {/solr,null,AVAILABLE} > >>> > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.AbstractConnector - > >>> > >>> Started ServerConnector@2ab5afc7{HTTP/1.1, > >>> > >>> (http/1.1)}{ > >>> 0.0.0.0:8188} > >>> > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.Server - > >>> > >>> Started > >>> > >>> @9626ms > >>> > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.AbstractConnector - > >>> > >>> Stopped ServerConnector@2ab5afc7{HTTP/1.1, > >>> > >>> (http/1.1)}{ > >>> 0.0.0.0:8188} > >>> > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.session > >>> > >>> - > >>> > >>> node0 > >>> > >>> Stopped > >>> > >>> scavenging > >>> [junit] [main] INFO > >>> > >>> org.eclipse.jetty.server.handler.ContextHandler - > >>> > >>> Stopped o.e.j.s.ServletContextHandler@7808f638 > >>> > >>> {/solr,null,STOPPED} > >>> > >>> ``` > >>> > >>> 2023年10月19日(木) 20:05 Mingchun Zhao< > >>> mingchun.zha...@gmail.com > >>> > >>> : > >>> > >>> Hi Karl, > >>> > >>> I've tried to update Kafka and its > >>> > >>> dependencies > >>> > >>> to > >>> > >>> the > >>> > >>> latest > >>> > >>> version > >>> > >>> including zookeeper, > >>> and confirmed tha kafka test run-IT-HSQLDB > >>> > >>> passed > >>> > >>> as > >>> > >>> below: > >>> > >>> ``` > >>> ~manifoldcf% cd connectors/kafka > >>> ~manifoldcf/connectors/kafka/% ant > >>> > >>> run-IT-HSQLDB > >>> > >>> BUILD SUCCESSFUL > >>> Total time: 1 minute 19 seconds > >>> ``` > >>> > >>> Also, I prepared a PR for this issue: > >>> > >>> https://github.com/apache/manifoldcf/pull/155 > >>> > >>> 2023年10月19日(木) 7:12 Karl Wright< > >>> daddy...@gmail.com>: > >>> > >>> Hi, > >>> > >>> It looks like the latest release of Kafka is > >>> > >>> 3.6.0. > >>> > >>> I'd try setting that version in the pom for > >>> > >>> connectors/kafka > >>> > >>> and > >>> > >>> doing > >>> > >>> mvn > >>> install. Then you can see what dependencies > >>> > >>> it > >>> > >>> wants > >>> > >>> by: > >>> > >>> mvn dependency:tree > >>> > >>> It may be that Kafka no longer even requires > >>> > >>> zookeeper > >>> > >>> - > >>> > >>> I > >>> > >>> didn't > >>> > >>> find > >>> > >>> it > >>> > >>> in a cursory inspection. But the > >>> > >>> dependency:tree > >>> > >>> would > >>> > >>> be > >>> > >>> the > >>> > >>> final > >>> > >>> word. > >>> > >>> Karl > >>> > >>> > >>> On Sat, Oct 14, 2023 at 2:17 AM Mingchun > >>> > >>> Zhao < > >>> mingchun.zha...@gmail.com > >>> > >>> wrote: > >>> > >>> > >>> Kalr, Thanks! > >>> Though I'm not familiar with kafka, I'll try > >>> > >>> to > >>> > >>> find > >>> > >>> out > >>> > >>> what's > >>> > >>> causing the > >>> > >>> error as much as possible. > >>> > >>> Kind Regards, > >>> Mingchun > >>> > >>> > >>> 2023年10月14日(土) 14:07 Karl Wright< > >>> daddy...@gmail.com > >>> > >>> : > >>> > >>> Yes, this seems to be something related to > >>> > >>> zookeeper > >>> > >>> update > >>> > >>> and > >>> > >>> the > >>> > >>> Kafka > >>> > >>> library version we're using. > >>> > >>> Someone will need to dig into what is going > >>> > >>> wrong > >>> > >>> here > >>> > >>> before > >>> > >>> we > >>> > >>> can > >>> > >>> release. I don't know how widely used the > >>> > >>> kafka > >>> > >>> connector is > >>> > >>> but > >>> > >>> if > >>> > >>> it > >>> > >>> is > >>> > >>> lightly used we can perhaps not distribute > >>> > >>> the > >>> > >>> connector > >>> > >>> any > >>> > >>> longer. But > >>> > >>> that would be a last choice. > >>> > >>> Karl > >>> > >>> > >>> On Fri, Oct 13, 2023 at 12:12 PM Mingchun > >>> > >>> Zhao > >>> > >>> < > >>> mingchun.zha...@gmail.com> > >>> > >>> wrote: > >>> > >>> > >>> By applying r1912939, I was able to > >>> > >>> confirm > >>> > >>> that > >>> > >>> the > >>> > >>> kafka > >>> > >>> test > >>> > >>> compile > >>> > >>> error has disappeared when running `ant > >>> > >>> test`. > >>> > >>> Thanks, Karl! > >>> > >>> However, the following error occurred on > >>> > >>> subsequent > >>> > >>> test > >>> > >>> runs. > >>> > >>> ``` > >>> [junit] Testsuite: > >>> > >>> > >>> org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT > >>> > >>> [junit] Tests run: 1, Failures: 0, Errors: > >>> > >>> 1, > >>> > >>> Skipped: > >>> > >>> 0, > >>> > >>> Time > >>> > >>> elapsed: > >>> > >>> 0 sec > >>> [junit] > >>> [junit] Testcase: > >>> > >>> > >>> > org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT:sanityCheck: > >>> > >>> Caused an ERROR > >>> [junit] Forked Java VM exited > >>> > >>> abnormally. > >>> > >>> Please > >>> > >>> note > >>> > >>> the > >>> > >>> time > >>> > >>> in > >>> > >>> the > >>> > >>> report does not reflect the time until the > >>> > >>> VM > >>> > >>> exit. > >>> > >>> [junit] > >>> > >>> junit.framework.AssertionFailedError: > >>> > >>> Forked > >>> > >>> Java > >>> > >>> VM > >>> > >>> exited > >>> > >>> abnormally. Please note the time in the > >>> > >>> report > >>> > >>> does > >>> > >>> not > >>> > >>> reflect > >>> > >>> the > >>> > >>> time > >>> > >>> until the VM exit. > >>> [junit] at > >>> > >>> > >>> jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown > >>> > >>> Source) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >>> > >>> [junit] at > >>> > >>> > >>> jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown > >>> > >>> Source) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >>> > >>> [junit] at > >>> > >>> > >>> jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown > >>> > >>> Source) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >>> > >>> [junit] at > >>> > >>> > >>> jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown > >>> > >>> Source) > >>> > >>> [junit] at > >>> > >>> > >>> > >>> > >>> > >>> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >>> > >>> [junit] > >>> [junit] > >>> > >>> BUILD FAILED > >>> > >>> > >>> /Users/zhaomingchun/ManifoldCF/manifoldcf/build.xml:517: > >>> > >>> The > >>> > >>> following > >>> > >>> error occurred while executing this line: > >>> > >>> > >>> /Users/zhaomingchun/ManifoldCF/manifoldcf/build.xml:471: > >>> > >>> The > >>> > >>> following > >>> > >>> error occurred while executing this line: > >>> > >>> > >>> > /Users/zhaomingchun/ManifoldCF/manifoldcf/dist/connector-build.xml:1102: > >>> > >>> Test > >>> > >>> org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT > >>> > >>> failed > >>> > >>> (crashed) > >>> ``` > >>> > >>> > >>> 2023年10月13日(金) 21:56 Karl Wright< > >>> daddy...@gmail.com > >>> > >>> : > >>> > >>> r1912939 fixes this but I need to spin a > >>> > >>> new > >>> > >>> RC. > >>> > >>> Karl > >>> > >>> > >>> On Fri, Oct 13, 2023 at 8:46 AM Karl > >>> > >>> Wright < > >>> daddy...@gmail.com> > >>> > >>> wrote: > >>> > >>> Yes I get the same thing; a test needs > >>> > >>> to > >>> > >>> be > >>> > >>> updated. > >>> > >>> [javac] > >>> > >>> > >>> > >>> > >>> > C:\wip\mcf\release-2.26-branch\connectors\kafka\connector\src\test\java\org\apache\manifoldcf\agents\output\kafka\ZooKeeperLocal.java:45: > >>> > >>> error: unreported exception > >>> > >>> AdminServerException; > >>> > >>> must > >>> > >>> be > >>> > >>> caught or > >>> > >>> declared to be thrown > >>> [javac] > >>> > >>> zooKeeperServer.runFromConfig(configuration); > >>> > >>> [javac] > >>> > >>> Karl > >>> > >>> > >>> On Fri, Oct 13, 2023 at 8:35 AM Karl > >>> > >>> Wright > >>> > >>> < > >>> daddy...@gmail.com> > >>> > >>> wrote: > >>> > >>> There was a Zookeeper dependency change > >>> > >>> this > >>> > >>> release. I > >>> > >>> wonder if > >>> > >>> there > >>> > >>> is a test that needs to be updated. > >>> > >>> Let > >>> > >>> me > >>> > >>> try > >>> > >>> and > >>> > >>> see. > >>> > >>> Karl > >>> > >>> > >>> On Fri, Oct 13, 2023 at 4:51 AM > >>> > >>> Piergiorgio > >>> > >>> Lucidi > >>> > >>> < > >>> piergior...@apache.org> wrote: > >>> > >>> > >>> Hi Mingchun, > >>> > >>> thank you for your message and I was > >>> > >>> trying > >>> > >>> to > >>> > >>> build > >>> > >>> ManifoldCF > >>> > >>> using > >>> > >>> OpenJDK 17 so probably in the future > >>> > >>> for > >>> > >>> supporting > >>> > >>> this > >>> > >>> version > >>> > >>> of > >>> > >>> Java > >>> > >>> we > >>> should include Jaxb libraries as well. > >>> > >>> The build is ok now and I can compile > >>> > >>> and > >>> > >>> package > >>> > >>> everything > >>> > >>> correctly. > >>> > >>> Unfortunately executing tests I have > >>> > >>> the > >>> > >>> following > >>> > >>> error: > >>> > >>> compile-tests: > >>> [javac] Compiling 1 source file > >>> > >>> to > >>> > >>> > >>> > >>> > /Users/piergiorgiolucidi/Downloads/apache-manifoldcf-2.26/connectors/kafka/build/connector-tests/classes > >>> > >>> [javac] > >>> > >>> > >>> > >>> > >>> > >>> > /Users/piergiorgiolucidi/Downloads/apache-manifoldcf-2.26/connectors/kafka/connector/src/test/java/org/apache/manifoldcf/agents/output/kafka/ZooKeeperLocal.java:45: > >>> > >>> error: unreported exception > >>> > >>> AdminServerException; > >>> > >>> must > >>> > >>> be > >>> > >>> caught > >>> > >>> or > >>> > >>> declared to be thrown > >>> [javac] > >>> > >>> zooKeeperServer.runFromConfig(configuration); > >>> > >>> [javac] > >>> > >>> ^ > >>> > >>> [javac] 1 error > >>> > >>> BUILD FAILED > >>> > >>> > >>> > /Users/piergiorgiolucidi/Downloads/apache-manifoldcf-2.26/build.xml:497: > >>> > >>> The following error occurred while > >>> > >>> executing > >>> > >>> this > >>> > >>> line: > >>> > >>> > /Users/piergiorgiolucidi/Downloads/apache-manifoldcf-2.26/build.xml:471: > >>> > >>> The following error occurred while > >>> > >>> executing > >>> > >>> this > >>> > >>> line: > >>> > >>> > >>> > >>> > /Users/piergiorgiolucidi/Downloads/apache-manifoldcf-2.26/dist/connector-build.xml:720: > >>> > >>> Compile failed; see the compiler error > >>> > >>> output > >>> > >>> for > >>> > >>> details. > >>> > >>> Any ideas? > >>> > >>> Thanks. > >>> > >>> Cheers, > >>> PG > >>> > >>> > >>> > >>> Il giorno gio 12 ott 2023 alle ore > >>> > >>> 10:39 > >>> > >>> Mingchun > >>> > >>> Zhao < > >>> mingchun.zha...@gmail.com> ha > >>> > >>> scritto: > >>> > >>> Hi Piergiorgio, > >>> > >>> FYI, Allow me to share the java and > >>> > >>> ant > >>> > >>> versions > >>> > >>> and > >>> > >>> the > >>> > >>> build > >>> > >>> steps > >>> > >>> I > >>> > >>> ran > >>> > >>> in my environment. > >>> > >>> ``` > >>> $ java --version > >>> openjdk 11.0.11 2021-04-20 > >>> OpenJDK Runtime Environment > >>> > >>> AdoptOpenJDK-11.0.11+9 > >>> > >>> (build > >>> > >>> 11.0.11+9) > >>> > >>> OpenJDK 64-Bit Server VM > >>> > >>> AdoptOpenJDK-11.0.11+9 > >>> > >>> (build > >>> > >>> 11.0.11+9, > >>> > >>> mixed > >>> > >>> mode) > >>> > >>> $ ant -version > >>> Apache Ant(TM) version 1.10.0 > >>> > >>> compiled > >>> > >>> on > >>> > >>> December > >>> > >>> 27 > >>> > >>> 2016 > >>> > >>> ``` > >>> ``` > >>> ant clean > >>> ant clean-deps > >>> ant clean-core-deps > >>> ant make-core-deps > >>> ant make-deps > >>> ant build > >>> ``` > >>> > >>> Regards, > >>> Mingchun > >>> > >>> 2023年10月12日(木) 17:32 Piergiorgio > >>> > >>> Lucidi < > >>> piergior...@apache.org > >>> > >>> : > >>> > >>> Hi folks, > >>> > >>> it seems that I can't compile the > >>> > >>> CswsConnector: > >>> > >>> [javac] public List<? extends > >>> > >>> AttributeGroup> > >>> > >>> getAttributeGroups() > >>> > >>> [javac] ^ > >>> [javac] symbol: class > >>> > >>> AttributeGroup > >>> > >>> [javac] location: class > >>> > >>> CswsConnector.ObjectInformation > >>> > >>> [javac] > >>> > >>> > >>> > >>> > >>> > >>> > /Volumes/BackupPJ/ManifoldCF-release/apache-manifoldcf-2.26/connectors/csws/connector/src/main/java/org/apache/manifoldcf/crawler/connectors/csws/CswsConnector.java:3966: > >>> > >>> error: cannot find symbol > >>> [javac] public > >>> > >>> NodePermissions > >>> > >>> getPermissions() > >>> > >>> [javac] ^ > >>> [javac] symbol: class > >>> > >>> NodePermissions > >>> > >>> [javac] location: class > >>> > >>> CswsConnector.ObjectInformation > >>> > >>> [javac] 100 errors > >>> [javac] 1 warning > >>> [javac] only showing the > >>> > >>> first > >>> > >>> 100 > >>> > >>> errors, > >>> > >>> of > >>> > >>> 123 > >>> > >>> total; > >>> > >>> use > >>> > >>> -Xmaxerrs > >>> > >>> if you would like to see more > >>> > >>> BUILD FAILED > >>> > >>> > >>> > >>> > >>> > /Volumes/BackupPJ/ManifoldCF-release/apache-manifoldcf-2.26/build.xml:489: > >>> > >>> The following error occurred while > >>> > >>> executing > >>> > >>> this > >>> > >>> line: > >>> > >>> > >>> > >>> > >>> > /Volumes/BackupPJ/ManifoldCF-release/apache-manifoldcf-2.26/build.xml:471: > >>> > >>> The following error occurred while > >>> > >>> executing > >>> > >>> this > >>> > >>> line: > >>> > >>> > >>> > >>> > /Volumes/BackupPJ/ManifoldCF-release/apache-manifoldcf-2.26/dist/connector-build.xml:686: > >>> > >>> Compile failed; see the compiler > >>> > >>> error > >>> > >>> output > >>> > >>> for > >>> > >>> details. > >>> > >>> Do you have any ideas? > >>> > >>> Thanks, > >>> PG > >>> > >>> Il giorno gio 12 ott 2023 alle ore > >>> > >>> 09:03 > >>> > >>> Guylaine > >>> > >>> BASSETTE < > >>> guylaine.basse...@francelabs.com> > >>> > >>> ha > >>> > >>> scritto: > >>> > >>> Hi, > >>> > >>> +1 from France Labs > >>> > >>> Regards, > >>> Guylaine > >>> > >>> France Labs – Your knowledge, now > >>> Datafari Enterprise Search – > >>> > >>> Découvrez > >>> > >>> la > >>> > >>> version > >>> > >>> 5 / > >>> > >>> Discover > >>> > >>> our > >>> > >>> version > >>> 5www.datafari.com< http://www.datafari.com> <http://www.datafari.com> > >>> > >>> Le 11/10/2023 à 23:12, Furkan > >>> > >>> KAMACI > >>> > >>> a > >>> > >>> écrit > >>> > >>> : > >>> > >>> Hi, > >>> > >>> +1! > >>> > >>> Kind regards, > >>> Furkan Kamaci > >>> > >>> On Wed, Oct 11, 2023 at 11:13 PM > >>> > >>> Mingchun > >>> > >>> Zhao< > >>> mingchun.zha...@gmail.com > >>> > >>> wrote: > >>> > >>> > >>> Hi, > >>> Ran all the tests. > >>> +1 from me. > >>> > >>> Kind regards, > >>> Mingchun > >>> > >>> 2023年10月12日(木) 4:29 Cihad Guzel< > >>> cguz...@gmail.com > >>> > >>> : > >>> > >>> Hi, > >>> > >>> +1 from me > >>> > >>> Regards, > >>> Cihad Guzel > >>> > >>> 11 Eki 2023 Çar, saat 15:08 > >>> > >>> tarihinde > >>> > >>> Karl > >>> > >>> Wright< > >>> daddy...@gmail.com > >>> > >>> şunu > >>> yazdı: > >>> > >>> > >>> Please vote on whether to > >>> > >>> release > >>> > >>> Apache > >>> > >>> ManifoldCF > >>> > >>> 2.26, > >>> > >>> RC0. > >>> > >>> This release is the first > >>> > >>> release > >>> > >>> that > >>> > >>> requires > >>> > >>> at > >>> > >>> least > >>> > >>> Java > >>> > >>> 11, > >>> > >>> and > >>> > >>> it > >>> > >>> also includes a new CSV > >>> > >>> connector > >>> > >>> along > >>> > >>> with > >>> > >>> support > >>> > >>> for > >>> > >>> Solr > >>> > >>> 9. > >>> > >>> The > >>> > >>> release artifact can be found > >>> > >>> at: > >>> > https://dist.apache.org/repos/dist/dev/manifoldcf/apache-manifoldcf-2.26 > >>> > >>> , > >>> > >>> and there is a release tag also > >>> > >>> at > >>> https://svn.apache.org/repos/asf/manifoldcf/tags/release-2.26-RC0 > >>> > >>> . > >>> > >>> Karl > >>> > >>> > >>> -- > >>> Piergiorgio > >>> > >>> > >>> -- > >>> Piergiorgio > >>> > >>> > >>> -- > >>> Cordialement, > >>> Guylaine > >>> > >>> France Labs – Your knowledge, now > >>> Datafari Enterprise Search – Découvrez la version > >>> > >>> 5 / > >>> > >>> Discover > >>> > >>> our > >>> > >>> version 5www.datafari.com<http://www.datafari.com> < > http://www.datafari.com> < http://www.datafari.com> > >>> > >>> -- > >>> Cordialement, > >>> Guylaine > >>> > >>> France Labs – Your knowledge, now > >>> Datafari Enterprise Search – Découvrez la version 5 / > >>> > >>> Discover > >>> > >>> our > >>> > >>> version > >>> 5www.datafari.com<http://www.datafari.com> <http://www.datafari.com> > <http://www.datafari.com> <http://www.datafari.com> > >>> > >>> -- > >>> Cordialement, > >>> Guylaine > >>> > >>> France Labs – Your knowledge, now > >>> Datafari Enterprise Search – Découvrez la version 5 / Discover our > >>> > >>> version > >>> 5www.datafari.com<http://www.datafari.com> <http://www.datafari.com> > <http://www.datafari.com> <http://www.datafari.com> > >>> > >>> -- > >>> Cordialement, > >>> Guylaine > >>> > >>> France Labs – Your knowledge, now > >>> Datafari Enterprise Search – Découvrez la version 5 / Discover our > >>> version 5www.datafari.com<http://www.datafari.com> < > http://www.datafari.com> < http://www.datafari.com> > >>> > >>> -- > >>> Cordialement, > >>> Guylaine > >>> > >>> France Labs – Your knowledge, now > >>> Datafari Enterprise Search – Découvrez la version 5 / Discover our > >>> > >>> version > >>> > >>> 5www.datafari.com > >>> > >>> -- > >>> Cordialement, > >>> Guylaine > >>> > >>> France Labs – Your knowledge, now > >>> Datafari Enterprise Search – Découvrez la version 5 / Discover our > >>> version 5 > >>> www.datafari.com > >>> > -- > Cordialement, > Guylaine > > France Labs – Your knowledge, now > Datafari Enterprise Search – Découvrez la version 5 / Discover our version > 5 > www.datafari.com <http://www.datafari.com>