I'm having trouble with the Solr test `ant run-IT-HSQLDB`.
It's been over an hour since I ran the test and it still doesn't finish.
It seems that the problem is probably due to the Manifold job not finishing.
Do you have any ideas or advice? Thanks!
```
~/ManifoldCF/manifoldcf/connectors/solr% ant run-IT-HSQLDB
Buildfile:
/Users/zhaomingchun/ManifoldCF/manifoldcf/connectors/solr/build.xml
... ...
run-IT-HSQLDB:
[junit] Testsuite:
org.apache.manifoldcf.agents.output.solr.tests.SolrCrawlHSQLDBIT
[junit] Configuration file successfully read
[junit] [main] INFO org.eclipse.jetty.util.log - Logging initialized
@7416ms to org.eclipse.jetty.util.log.Slf4jLog
[junit] [main] INFO org.eclipse.jetty.server.Server -
jetty-9.4.48.v20220622; built: 2022-06-21T20:42:25.880Z; git:
6b67c5719d1f4371b33655ff2d047d24e171e49a; jvm 11.0.11+9
[junit] [main] INFO org.eclipse.jetty.server.session -
DefaultSessionIdManager workerName=node0
[junit] [main] INFO org.eclipse.jetty.server.session - No
SessionScavenger set, using defaults
[junit] [main] INFO org.eclipse.jetty.server.session - node0 Scavenging
every 600000ms
[junit] [main] INFO org.eclipse.jetty.server.handler.ContextHandler -
Started o.e.j.w.WebAppContext@4bdc8b5d{ManifoldCF Crawler
Interface,/mcf-crawler-ui,file:///private/var/folders/zh/mx4q_qh93cv6jtp13ht8b1fr0000gn/T/jetty-0_0_0_0-8346-mcf-crawler-ui_war-_mcf-crawler-ui-any-17189990303852051874/webapp/,AVAILABLE}{/Users/zhaomingchun/ManifoldCF/manifoldcf/dist/web/war/mcf-crawler-ui.war}
[junit] [main] INFO org.eclipse.jetty.server.handler.ContextHandler -
Started o.e.j.w.WebAppContext@3bcd426c{ManifoldCF Authorities API
Webapp,/mcf-authority-service,file:///private/var/folders/zh/mx4q_qh93cv6jtp13ht8b1fr0000gn/T/jetty-0_0_0_0-8346-mcf-authority-service_war-_mcf-authority-service-any-1541416273384984853/webapp/,AVAILABLE}{/Users/zhaomingchun/ManifoldCF/manifoldcf/dist/web/war/mcf-authority-service.war}
[junit] Creating mock service
[junit] [main] INFO org.eclipse.jetty.server.handler.ContextHandler -
Started o.e.j.w.WebAppContext@5f14a673{ManifoldCF General API
Webapp,/mcf-api-service,file:///private/var/folders/zh/mx4q_qh93cv6jtp13ht8b1fr0000gn/T/jetty-0_0_0_0-8346-mcf-api-service_war-_mcf-api-service-any-12421572948842353797/webapp/,AVAILABLE}{/Users/zhaomingchun/ManifoldCF/manifoldcf/dist/web/war/mcf-api-service.war}
[junit] Mock service created
[junit] [main] INFO org.eclipse.jetty.server.AbstractConnector -
Started ServerConnector@70325d20{HTTP/1.1, (http/1.1)}{0.0.0.0:8346}
[junit] [main] INFO org.eclipse.jetty.server.Server - Started @9616ms
[junit] [main] INFO org.eclipse.jetty.server.Server -
jetty-9.4.48.v20220622; built: 2022-06-21T20:42:25.880Z; git:
6b67c5719d1f4371b33655ff2d047d24e171e49a; jvm 11.0.11+9
[junit] [main] INFO org.eclipse.jetty.server.session -
DefaultSessionIdManager workerName=node0
[junit] [main] INFO org.eclipse.jetty.server.session - No
SessionScavenger set, using defaults
[junit] [main] INFO org.eclipse.jetty.server.session - node0 Scavenging
every 600000ms
[junit] [main] INFO org.eclipse.jetty.server.handler.ContextHandler -
Started o.e.j.s.ServletContextHandler@7808f638{/solr,null,AVAILABLE}
[junit] [main] INFO org.eclipse.jetty.server.AbstractConnector -
Started ServerConnector@2ab5afc7{HTTP/1.1, (http/1.1)}{0.0.0.0:8188}
[junit] [main] INFO org.eclipse.jetty.server.Server - Started @9626ms
[junit] [main] INFO org.eclipse.jetty.server.AbstractConnector -
Stopped ServerConnector@2ab5afc7{HTTP/1.1, (http/1.1)}{0.0.0.0:8188}
[junit] [main] INFO org.eclipse.jetty.server.session - node0 Stopped
scavenging
[junit] [main] INFO org.eclipse.jetty.server.handler.ContextHandler -
Stopped o.e.j.s.ServletContextHandler@7808f638{/solr,null,STOPPED}
```
2023年10月19日(木) 20:05 Mingchun Zhao <mingchun.zha...@gmail.com>:
> Hi Karl,
>
> I've tried to update Kafka and its dependencies to the latest version
> including zookeeper,
> and confirmed tha kafka test run-IT-HSQLDB passed as below:
>
> ```
> ~manifoldcf% cd connectors/kafka
> ~manifoldcf/connectors/kafka/% ant run-IT-HSQLDB
>
> BUILD SUCCESSFUL
> Total time: 1 minute 19 seconds
> ```
>
> Also, I prepared a PR for this issue:
> https://github.com/apache/manifoldcf/pull/155
>
> 2023年10月19日(木) 7:12 Karl Wright <daddy...@gmail.com>:
>
>> Hi,
>>
>> It looks like the latest release of Kafka is 3.6.0.
>>
>> I'd try setting that version in the pom for connectors/kafka and doing mvn
>> install. Then you can see what dependencies it wants by:
>> mvn dependency:tree
>>
>> It may be that Kafka no longer even requires zookeeper - I didn't find it
>> in a cursory inspection. But the dependency:tree would be the final word.
>>
>> Karl
>>
>>
>> On Sat, Oct 14, 2023 at 2:17 AM Mingchun Zhao <mingchun.zha...@gmail.com>
>> wrote:
>>
>> > Kalr, Thanks!
>> > Though I'm not familiar with kafka, I'll try to find out what's causing
>> the
>> > error as much as possible.
>> >
>> > Kind Regards,
>> > Mingchun
>> >
>> >
>> > 2023年10月14日(土) 14:07 Karl Wright <daddy...@gmail.com>:
>> >
>> > > Yes, this seems to be something related to zookeeper update and the
>> Kafka
>> > > library version we're using.
>> > >
>> > > Someone will need to dig into what is going wrong here before we can
>> > > release. I don't know how widely used the kafka connector is but if
>> it
>> > is
>> > > lightly used we can perhaps not distribute the connector any longer.
>> But
>> > > that would be a last choice.
>> > >
>> > > Karl
>> > >
>> > >
>> > > On Fri, Oct 13, 2023 at 12:12 PM Mingchun Zhao <
>> > mingchun.zha...@gmail.com>
>> > > wrote:
>> > >
>> > > > By applying r1912939, I was able to confirm that the kafka test
>> compile
>> > > > error has disappeared when running `ant test`.
>> > > > Thanks, Karl!
>> > > >
>> > > > However, the following error occurred on subsequent test runs.
>> > > > ```
>> > > > [junit] Testsuite:
>> > > > org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT
>> > > > [junit] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time
>> > > elapsed:
>> > > > 0 sec
>> > > > [junit]
>> > > > [junit] Testcase:
>> > > >
>> > org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT:sanityCheck:
>> > > > Caused an ERROR
>> > > > [junit] Forked Java VM exited abnormally. Please note the time
>> in
>> > the
>> > > > report does not reflect the time until the VM exit.
>> > > > [junit] junit.framework.AssertionFailedError: Forked Java VM
>> exited
>> > > > abnormally. Please note the time in the report does not reflect the
>> > time
>> > > > until the VM exit.
>> > > > [junit] at
>> > > > jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>> > > > [junit] at
>> > > >
>> > > >
>> > >
>> >
>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> > > > [junit] at
>> > > > jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>> > > > [junit] at
>> > > >
>> > > >
>> > >
>> >
>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> > > > [junit] at
>> > > > jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>> > > > [junit] at
>> > > >
>> > > >
>> > >
>> >
>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> > > > [junit] at
>> > > > jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>> > > > [junit] at
>> > > >
>> > > >
>> > >
>> >
>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> > > > [junit]
>> > > > [junit]
>> > > >
>> > > > BUILD FAILED
>> > > > /Users/zhaomingchun/ManifoldCF/manifoldcf/build.xml:517: The
>> following
>> > > > error occurred while executing this line:
>> > > > /Users/zhaomingchun/ManifoldCF/manifoldcf/build.xml:471: The
>> following
>> > > > error occurred while executing this line:
>> > > >
>> > /Users/zhaomingchun/ManifoldCF/manifoldcf/dist/connector-build.xml:1102:
>> > > > Test org.apache.manifoldcf.agents.output.kafka.APISanityHSQLDBIT
>> failed
>> > > > (crashed)
>> > > > ```
>> > > >
>> > > >
>> > > > 2023年10月13日(金) 21:56 Karl Wright <daddy...@gmail.com>:
>> > > >
>> > > > > r1912939 fixes this but I need to spin a new RC.
>> > > > > Karl
>> > > > >
>> > > > >
>> > > > > On Fri, Oct 13, 2023 at 8:46 AM Karl Wright <daddy...@gmail.com>
>> > > wrote:
>> > > > >
>> > > > > > Yes I get the same thing; a test needs to be updated.
>> > > > > >
>> > > > > > [javac]
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> C:\wip\mcf\release-2.26-branch\connectors\kafka\connector\src\test\java\org\apache\manifoldcf\agents\output\kafka\ZooKeeperLocal.java:45:
>> > > > > > error: unreported exception AdminServerException; must be
>> caught or
>> > > > > > declared to be thrown
>> > > > > > [javac]
>> zooKeeperServer.runFromConfig(configuration);
>> > > > > > [javac]
>> > > > > >
>> > > > > > Karl
>> > > > > >
>> > > > > >
>> > > > > > On Fri, Oct 13, 2023 at 8:35 AM Karl Wright <daddy...@gmail.com
>> >
>> > > > wrote:
>> > > > > >
>> > > > > >> There was a Zookeeper dependency change this release. I
>> wonder if
>> > > > there
>> > > > > >> is a test that needs to be updated. Let me try and see.
>> > > > > >>
>> > > > > >> Karl
>> > > > > >>
>> > > > > >>
>> > > > > >> On Fri, Oct 13, 2023 at 4:51 AM Piergiorgio Lucidi <
>> > > > > >> piergior...@apache.org> wrote:
>> > > > > >>
>> > > > > >>> Hi Mingchun,
>> > > > > >>>
>> > > > > >>> thank you for your message and I was trying to build
>> ManifoldCF
>> > > using
>> > > > > >>> OpenJDK 17 so probably in the future for supporting this
>> version
>> > of
>> > > > > Java
>> > > > > >>> we
>> > > > > >>> should include Jaxb libraries as well.
>> > > > > >>>
>> > > > > >>> The build is ok now and I can compile and package everything
>> > > > correctly.
>> > > > > >>> Unfortunately executing tests I have the following error:
>> > > > > >>>
>> > > > > >>> compile-tests:
>> > > > > >>> [javac] Compiling 1 source file to
>> > > > > >>>
>> > > > > >>>
>> > > > >
>> > > >
>> > >
>> >
>> /Users/piergiorgiolucidi/Downloads/apache-manifoldcf-2.26/connectors/kafka/build/connector-tests/classes
>> > > > > >>> [javac]
>> > > > > >>>
>> > > > > >>>
>> > > > >
>> > > >
>> > >
>> >
>> /Users/piergiorgiolucidi/Downloads/apache-manifoldcf-2.26/connectors/kafka/connector/src/test/java/org/apache/manifoldcf/agents/output/kafka/ZooKeeperLocal.java:45:
>> > > > > >>> error: unreported exception AdminServerException; must be
>> caught
>> > or
>> > > > > >>> declared to be thrown
>> > > > > >>> [javac]
>> > zooKeeperServer.runFromConfig(configuration);
>> > > > > >>> [javac] ^
>> > > > > >>> [javac] 1 error
>> > > > > >>>
>> > > > > >>> BUILD FAILED
>> > > > > >>>
>> > > > >
>> > >
>> /Users/piergiorgiolucidi/Downloads/apache-manifoldcf-2.26/build.xml:497:
>> > > > > >>> The following error occurred while executing this line:
>> > > > > >>>
>> > > > >
>> > >
>> /Users/piergiorgiolucidi/Downloads/apache-manifoldcf-2.26/build.xml:471:
>> > > > > >>> The following error occurred while executing this line:
>> > > > > >>>
>> > > > > >>>
>> > > > >
>> > > >
>> > >
>> >
>> /Users/piergiorgiolucidi/Downloads/apache-manifoldcf-2.26/dist/connector-build.xml:720:
>> > > > > >>> Compile failed; see the compiler error output for details.
>> > > > > >>>
>> > > > > >>> Any ideas?
>> > > > > >>>
>> > > > > >>> Thanks.
>> > > > > >>>
>> > > > > >>> Cheers,
>> > > > > >>> PG
>> > > > > >>>
>> > > > > >>>
>> > > > > >>>
>> > > > > >>> Il giorno gio 12 ott 2023 alle ore 10:39 Mingchun Zhao <
>> > > > > >>> mingchun.zha...@gmail.com> ha scritto:
>> > > > > >>>
>> > > > > >>> > Hi Piergiorgio,
>> > > > > >>> >
>> > > > > >>> > FYI, Allow me to share the java and ant versions and the
>> build
>> > > > steps
>> > > > > I
>> > > > > >>> ran
>> > > > > >>> > in my environment.
>> > > > > >>> >
>> > > > > >>> > ```
>> > > > > >>> > $ java --version
>> > > > > >>> > openjdk 11.0.11 2021-04-20
>> > > > > >>> > OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9 (build
>> > > > 11.0.11+9)
>> > > > > >>> > OpenJDK 64-Bit Server VM AdoptOpenJDK-11.0.11+9 (build
>> > 11.0.11+9,
>> > > > > mixed
>> > > > > >>> > mode)
>> > > > > >>> >
>> > > > > >>> > $ ant -version
>> > > > > >>> > Apache Ant(TM) version 1.10.0 compiled on December 27 2016
>> > > > > >>> > ```
>> > > > > >>> > ```
>> > > > > >>> > ant clean
>> > > > > >>> > ant clean-deps
>> > > > > >>> > ant clean-core-deps
>> > > > > >>> > ant make-core-deps
>> > > > > >>> > ant make-deps
>> > > > > >>> > ant build
>> > > > > >>> > ```
>> > > > > >>> >
>> > > > > >>> > Regards,
>> > > > > >>> > Mingchun
>> > > > > >>> >
>> > > > > >>> > 2023年10月12日(木) 17:32 Piergiorgio Lucidi <
>> > piergior...@apache.org
>> > > >:
>> > > > > >>> >
>> > > > > >>> > > Hi folks,
>> > > > > >>> > >
>> > > > > >>> > > it seems that I can't compile the CswsConnector:
>> > > > > >>> > >
>> > > > > >>> > > [javac] public List<? extends AttributeGroup>
>> > > > > >>> getAttributeGroups()
>> > > > > >>> > > [javac] ^
>> > > > > >>> > > [javac] symbol: class AttributeGroup
>> > > > > >>> > > [javac] location: class
>> CswsConnector.ObjectInformation
>> > > > > >>> > > [javac]
>> > > > > >>> > >
>> > > > > >>> > >
>> > > > > >>> >
>> > > > > >>>
>> > > > >
>> > > >
>> > >
>> >
>> /Volumes/BackupPJ/ManifoldCF-release/apache-manifoldcf-2.26/connectors/csws/connector/src/main/java/org/apache/manifoldcf/crawler/connectors/csws/CswsConnector.java:3966:
>> > > > > >>> > > error: cannot find symbol
>> > > > > >>> > > [javac] public NodePermissions getPermissions()
>> > > > > >>> > > [javac] ^
>> > > > > >>> > > [javac] symbol: class NodePermissions
>> > > > > >>> > > [javac] location: class
>> CswsConnector.ObjectInformation
>> > > > > >>> > > [javac] 100 errors
>> > > > > >>> > > [javac] 1 warning
>> > > > > >>> > > [javac] only showing the first 100 errors, of 123
>> total;
>> > > use
>> > > > > >>> > -Xmaxerrs
>> > > > > >>> > > if you would like to see more
>> > > > > >>> > >
>> > > > > >>> > > BUILD FAILED
>> > > > > >>> > >
>> > > > > >>> >
>> > > > > >>>
>> > > > >
>> > > >
>> > >
>> >
>> /Volumes/BackupPJ/ManifoldCF-release/apache-manifoldcf-2.26/build.xml:489:
>> > > > > >>> > > The following error occurred while executing this line:
>> > > > > >>> > >
>> > > > > >>> >
>> > > > > >>>
>> > > > >
>> > > >
>> > >
>> >
>> /Volumes/BackupPJ/ManifoldCF-release/apache-manifoldcf-2.26/build.xml:471:
>> > > > > >>> > > The following error occurred while executing this line:
>> > > > > >>> > >
>> > > > > >>> > >
>> > > > > >>> >
>> > > > > >>>
>> > > > >
>> > > >
>> > >
>> >
>> /Volumes/BackupPJ/ManifoldCF-release/apache-manifoldcf-2.26/dist/connector-build.xml:686:
>> > > > > >>> > > Compile failed; see the compiler error output for details.
>> > > > > >>> > >
>> > > > > >>> > > Do you have any ideas?
>> > > > > >>> > >
>> > > > > >>> > > Thanks,
>> > > > > >>> > > PG
>> > > > > >>> > >
>> > > > > >>> > > Il giorno gio 12 ott 2023 alle ore 09:03 Guylaine
>> BASSETTE <
>> > > > > >>> > > guylaine.basse...@francelabs.com> ha scritto:
>> > > > > >>> > >
>> > > > > >>> > > > Hi,
>> > > > > >>> > > >
>> > > > > >>> > > > +1 from France Labs
>> > > > > >>> > > >
>> > > > > >>> > > > Regards,
>> > > > > >>> > > > Guylaine
>> > > > > >>> > > >
>> > > > > >>> > > > France Labs – Your knowledge, now
>> > > > > >>> > > > Datafari Enterprise Search – Découvrez la version 5 /
>> > > Discover
>> > > > > our
>> > > > > >>> > > version
>> > > > > >>> > > > 5
>> > > > > >>> > > > www.datafari.com <http://www.datafari.com>
>> > > > > >>> > > > Le 11/10/2023 à 23:12, Furkan KAMACI a écrit :
>> > > > > >>> > > > > Hi,
>> > > > > >>> > > > >
>> > > > > >>> > > > > +1!
>> > > > > >>> > > > >
>> > > > > >>> > > > > Kind regards,
>> > > > > >>> > > > > Furkan Kamaci
>> > > > > >>> > > > >
>> > > > > >>> > > > > On Wed, Oct 11, 2023 at 11:13 PM Mingchun Zhao<
>> > > > > >>> > > mingchun.zha...@gmail.com
>> > > > > >>> > > > >
>> > > > > >>> > > > > wrote:
>> > > > > >>> > > > >
>> > > > > >>> > > > >> Hi,
>> > > > > >>> > > > >> Ran all the tests.
>> > > > > >>> > > > >> +1 from me.
>> > > > > >>> > > > >>
>> > > > > >>> > > > >> Kind regards,
>> > > > > >>> > > > >> Mingchun
>> > > > > >>> > > > >>
>> > > > > >>> > > > >> 2023年10月12日(木) 4:29 Cihad Guzel<cguz...@gmail.com>:
>> > > > > >>> > > > >>
>> > > > > >>> > > > >>> Hi,
>> > > > > >>> > > > >>>
>> > > > > >>> > > > >>> +1 from me
>> > > > > >>> > > > >>>
>> > > > > >>> > > > >>> Regards,
>> > > > > >>> > > > >>> Cihad Guzel
>> > > > > >>> > > > >>>
>> > > > > >>> > > > >>> 11 Eki 2023 Çar, saat 15:08 tarihinde Karl Wright<
>> > > > > >>> > daddy...@gmail.com
>> > > > > >>> > > >
>> > > > > >>> > > > >>> şunu
>> > > > > >>> > > > >>> yazdı:
>> > > > > >>> > > > >>>
>> > > > > >>> > > > >>>> Please vote on whether to release Apache ManifoldCF
>> > > 2.26,
>> > > > > RC0.
>> > > > > >>> > > > >>>> This release is the first release that requires at
>> > least
>> > > > > Java
>> > > > > >>> 11,
>> > > > > >>> > > and
>> > > > > >>> > > > >> it
>> > > > > >>> > > > >>>> also includes a new CSV connector along with
>> support
>> > for
>> > > > > Solr
>> > > > > >>> 9.
>> > > > > >>> > > The
>> > > > > >>> > > > >>>> release artifact can be found at:
>> > > > > >>> > > > >>>>
>> > > > > >>> > > > >>
>> > > > > >>> > > >
>> > > > > >>> >
>> > > > > >>>
>> > > > >
>> > >
>> https://dist.apache.org/repos/dist/dev/manifoldcf/apache-manifoldcf-2.26
>> > > > > >>> > > > >>> ,
>> > > > > >>> > > > >>>> and there is a release tag also at
>> > > > > >>> > > > >>>>
>> > > > > >>>
>> > https://svn.apache.org/repos/asf/manifoldcf/tags/release-2.26-RC0
>> > > > > >>> > > .
>> > > > > >>> > > > >>>>
>> > > > > >>> > > > >>>> Karl
>> > > > > >>> > > > >>>>
>> > > > > >>> > >
>> > > > > >>> > >
>> > > > > >>> > >
>> > > > > >>> > > --
>> > > > > >>> > > Piergiorgio
>> > > > > >>> > >
>> > > > > >>> >
>> > > > > >>>
>> > > > > >>>
>> > > > > >>> --
>> > > > > >>> Piergiorgio
>> > > > > >>>
>> > > > > >>
>> > > > >
>> > > >
>> > >
>> >
>>
>
