Re: newbie questions about UIMA Types
Thanks for the detailed reply and examples. I¹ve got some tangentially related questions about types in UIMA C++, which I hope that either you or someone else can answer: If you need to use a custom Type in an annotator written with the UIMA C++ SDK, 1) do you need to define a corresponding custom C++ class (analogous to the one generated by JCasGen)? 2) if so, is there a comparable CppCasGen, or do you need to write it manually? Thanks in advance, David On 11/30/16, 8:23 PM, "Richard Eckart de Castilho" <r...@apache.org> wrote: >It is possible to customize the generated JCas classes, yes. You can e.g. >add own methods or even own fields. However, own fields would not be >saved/loaded when you persist a CAS e.g. to XMI. > >As a case for a custom method, consider e.g. the DKPro Core Token >"setText(string)" method [1]. >If the "string" passed to the method differs from the covered text of the >Token, then a new >"Form" annotation with the value "string" is created, linked to the Token. > >Another case would be the "links()" method on the DKPro Core CorefChain >type. It returns all >elements in the respective coreference chain as a List thus saving the >user to manually iterate >over the whole chain to reach all elements. > >FSList and friends are built-in types of UIMA Core - you can't modify >these. But uimaFIT provides >several methods to make working with these things much more convenient. >See > >- org.apache.uima.fit.util.FSCollectionFactory and its methods to create >FSList etc from Java collections >- org.apache.uima.fit.util.JCasUtil has select methods to retrieve >elements from FSList etc >- org.apache.uima.fit.util.FSUtil has methods to conveniently get/set >feature values including multi-valued features. > >Best, > >-- Richard > >[1] >https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_dkpro_dkpr >o-2Dcore_blob_71fda5c6ba91748b6e87312554e418ac1e2911c6_dkpro-2Dcore-2Dapi- >2Dsegmentation-2Dasl_src_main_java_de_tudarmstadt_ukp_dkpro_core_api_segme >ntation_type_Token.java-23L313=DgIF-g=3XrKki35ZWuh8X2qbeRISQ=BYS7q6K >6Famz8NiMJzvOgYA-WQSvBt9z6TEbaT3nnNM=HngGj3axgoDuVIMZym8FO61Tu_FMjQ_zxdk >T4SVvZWQ=XrClIvXlvCk4wq9FakxA9hWNOdyZAcmxRvmyBj9GJaw= >[2] >https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_dkpro_dkpr >o-2Dcore_blob_ba33629fc0f077337f9af39e38e1b58531e1674e_dkpro-2Dcore-2Dapi- >2Dcoref-2Dasl_src_main_java_de_tudarmstadt_ukp_dkpro_core_api_coref_type_C >oreferenceChain.java-23L101=DgIF-g=3XrKki35ZWuh8X2qbeRISQ=BYS7q6K6Fa >mz8NiMJzvOgYA-WQSvBt9z6TEbaT3nnNM=HngGj3axgoDuVIMZym8FO61Tu_FMjQ_zxdkT4S >VvZWQ=ul8Zztzk4X2HysLTy5P9MA6G_SHnU-firAU3B9s9EMc= > >> On 30.11.2016, at 20:25, David Fox <david@humedica.com> wrote: >> >> Does the UIMA Java framework support modifying or extend the java class >>generated by JCasGen corresponding to a custom Type? If so, are there >>any common circumstances where this is necessary? >> >> I didn¹t see anything in the examples or documentation about modifying >>the generated classes, but I also didn¹t see anything saying you >>couldn¹t. I suspect that this is not supported (and that otherwise you >>wouldn¹t be able to pass a CAS between distributed UIMA AS components, >>or between a Java annotator and a C++ one). But it would be nice to >>know for certain. >> >> The reason I ask is that the set of data structures supported by UIMA >>types (individual FS references, FSList linked lists, and FSArray >>arrays) is fairly limited compared to modern programming languages, >>which often directly support associative arrays, trees, and graphs. I¹m >>trying to understand whether this is a restriction on the implementation >>of custom types (which it would be if modifying/extending the generated >>class was not supported), or just on the public interface accessible via >>the UIMA API. >> >> David
newbie questions about UIMA Types
Does the UIMA Java framework support modifying or extend the java class generated by JCasGen corresponding to a custom Type? If so, are there any common circumstances where this is necessary? I didn’t see anything in the examples or documentation about modifying the generated classes, but I also didn’t see anything saying you couldn’t. I suspect that this is not supported (and that otherwise you wouldn’t be able to pass a CAS between distributed UIMA AS components, or between a Java annotator and a C++ one). But it would be nice to know for certain. The reason I ask is that the set of data structures supported by UIMA types (individual FS references, FSList linked lists, and FSArray arrays) is fairly limited compared to modern programming languages, which often directly support associative arrays, trees, and graphs. I’m trying to understand whether this is a restriction on the implementation of custom types (which it would be if modifying/extending the generated class was not supported), or just on the public interface accessible via the UIMA API. David
Re: UIMA C++ in a Mac
I tried this a couple of months back and got the same error. Eventually I gave up on Mac OS X and got it working on linux. In our case, we use Mac OS X laptops for testing, but our production environment is linux, so this wasn¹t a big deal and I didn¹t pursue it further. If I really did need to test on Mac OS X, my plan was to install virtual box and run a linux VM on my Mac laptop. David On 11/25/16, 5:07 PM, "William Colen"wrote: >Hi, > >I need to execute a NLTK based POS Tagger. I was trying to create a C++ AE >for that. Do you think it will work? > >I installed UIMA C++ 2.4.0 on top of UIMA 2.9.0. > >When I try to execute the installation instruction I get some compilation >errors, like follows: > >$ make -f ExampleApplication.mak >g++ -Wall -x c++ -DDEBUG -g -fno-inline -fPIC -fno-default-inline >-I/Users/user/programs/apache-uima/uimacpp/include >-I/Users/user/programs/apache-uima/uimacpp/include/apr-1 -Wno-deprecated >-c ExampleApplication.cpp >clang: warning: optimization flag '-fno-default-inline' is not supported >In file included from ExampleApplication.cpp:20: >In file included from >/Users/user/programs/apache-uima/uimacpp/include/uima/api.hpp:36: >In file included from >/Users/user/programs/apache-uima/uimacpp/include/uima/annotator_context.hp >p:48: >In file included from >/Users/user/programs/apache-uima/uimacpp/include/uima/engine.hpp:47: >In file included from >/Users/user/programs/apache-uima/uimacpp/include/uima/exceptions.hpp:88: >/Users/user/programs/apache-uima/uimacpp/include/uima/strtools.hpp:731:14: >error: no member named 'operator void *' in >'std::__1::basic_istream' > if (!i.operator void*()) > ~ ^ >/Users/user/programs/apache-uima/uimacpp/include/uima/strtools.hpp:737:18: >error: no member named 'operator void *' in >'std::__1::basic_istream' > if (!i.operator void*()) > ~ ^ >/Users/user/programs/apache-uima/uimacpp/include/uima/strtools.hpp:1674:19 >: >warning: comparison of constant 18446744073709551615 with expression of >type 'unsigned int' is always true >[-Wtautological-constant-out-of-range-compare] >if (envBegPos != STRING_NPOS) { >~ ^ ~~~ >/Users/user/programs/apache-uima/uimacpp/include/uima/strtools.hpp:1676:21 >: >warning: comparison of constant 18446744073709551615 with expression of >type 'unsigned int' is always true >[-Wtautological-constant-out-of-range-compare] > if (envEndPos != STRING_NPOS) { > ~ ^ ~~~ >In file included from ExampleApplication.cpp:20: >In file included from >/Users/user/programs/apache-uima/uimacpp/include/uima/api.hpp:36: >In file included from >/Users/user/programs/apache-uima/uimacpp/include/uima/annotator_context.hp >p:48: >In file included from >/Users/user/programs/apache-uima/uimacpp/include/uima/engine.hpp:50: >In file included from >/Users/user/programs/apache-uima/uimacpp/include/uima/casiterator.hpp:37: >/Users/user/programs/apache-uima/uimacpp/include/uima/cas.hpp:1252:84: >error: default argument references parameter 'enAmbiguous' >ANIterator subIterator( Type const & crType, EnIteratorAmbiguity >enAmbiguous = enAmbiguous ) const; > > ^~~ >/Users/user/programs/apache-uima/uimacpp/include/uima/cas.hpp:1376:88: >error: default argument references parameter 'enAmbiguous' >ANIterator subIterator( AnnotationFS const & an, EnIteratorAmbiguity >enAmbiguous = enAmbiguous ) const; > > ^~~ >ExampleApplication.cpp:207:12: warning: unused variable 'numread' >[-Wunused-variable] >size_t numread = fread(pBuffer,1,filesize,pFile); > ^ >In file included from ExampleApplication.cpp:20: >In file included from >/Users/user/programs/apache-uima/uimacpp/include/uima/api.hpp:59: >/Users/user/programs/apache-uima/uimacpp/include/uima/annotator_abase.hpp: >138:24: >warning: private field 'iv_typeSystem' is not used >[-Wunused-private-field] >TypeSystem const * iv_typeSystem; > ^ >4 warnings and 4 errors generated. >make: *** [ExampleApplication.o] Error 1 > > > > >What I am missing? > >Thank you, >William
Re: NullPointerException when RunRemoteAsyncAE tries to unload (UIMA AS and UIMA C++)
That’s odd… now I get java.io.IOException: Error: no socket connection. at org.apache.uima.aae.controller.UimacppServiceManagement.quiesceAndStop(Uima cppServiceManagement.java:109) at org.apache.uima.aae.controller.UimacppServiceManagement.quiesceAndShutdown( UimacppServiceManagement.java:409) at org.apache.uima.aae.controller.UimacppShutdownHook.run(UimacppServiceContro ller.java:1105) UimacppShutdownHook destroy C++ process I can’t tell for sure, because the source code in trunk doesn’t match this latest snapshot, but it looks as if I’m now hitting the else clause from the if statement containing the code which was generating the previous exception. Full output below: Attempting to deploy descriptors/Deploy_MeetingAnnotator.xml ... Waiting for Uima C++ service to connect... Waiting for Uima C++ service to connect... Waiting for Uima C++ service to report init status... deployCppService.cpp91 Start receiving messages Startinging GetMetaData instance Starting Annotator instance 0 deployCppService.cpp UIMA C++ Service MeetingAnnotator at tcp://localhost:61616 Ready to process... Uima C++ service at MeetingAnnotator Ready to process... UIMA AS Service Initialization Complete Completed 8 documents; 17165 characters Time Elapsed : 6262 ms 0 8 apr_socket_recv command=SHUTDOWN deployCppService.cpp Shutdown started. Stopping GetMetaData instance Stopping Annotator instance 0 deployCppService.cpp Shutdown done. UimacppServiceManagement service reports shutdown DONE Error no connection UimacppShutdownHook sending quiesce message java.io.IOException: Error: no socket connection. at org.apache.uima.aae.controller.UimacppServiceManagement.quiesceAndStop(Uima cppServiceManagement.java:109) at org.apache.uima.aae.controller.UimacppServiceManagement.quiesceAndShutdown( UimacppServiceManagement.java:409) at org.apache.uima.aae.controller.UimacppShutdownHook.run(UimacppServiceContro ller.java:1105) UimacppShutdownHook destroy C++ process On 9/23/16, 11:31 AM, "Jaroslaw Cwiklik" <uim...@gmail.com> wrote: >David, here is another uima-as snapshot for you to test > >https://urldefense.proofpoint.com/v2/url?u=https-3A__repository.apache.org >_content_repositories_snapshots_org_apache_uima_uima-2Das_2.9.0-2DSNAPSHOT >_uima-2Das-2D2.9.0-2D20160923.152659-2D5.tar.gz=DQIFaQ=3XrKki35ZWuh8X2 >qbeRISQ=BYS7q6K6Famz8NiMJzvOgYA-WQSvBt9z6TEbaT3nnNM=tXj0BK0pkaseF8BQPk >gw2nEjl9UxDSq736aXsJcDJ0g=iB7ah2MjAK4kl3Ops3aKk9yYcmxk19IcBDuWHgkGN5U= > > >I have not tested the fix. All I did was to modify quiesceAndStop() and >terminate() to catch SocketException and log a message. This code no >longer dumps a stack trace nor re-throws an exception. > >Let me know if this addresses the problem and I will close the JIRA > >-jerry > >On Fri, Sep 23, 2016 at 10:24 AM, Jaroslaw Cwiklik <uim...@gmail.com> >wrote: > >> David, so there is progress. :) >> >> I am not familiar with UimacppServiceController code but a quick glance >>at >> the source >> suggests that this code communicates with a C++ process via a socket: >> >> >> if (socket != null) { >> // System.out.println("UimacppServiceManagement::quiesceAndStop() >> // Sending QUIESCEANDSTOP"); >> writer.write("QUIESCEANDSTOP"); >> writer.flush(); >> BufferedReader in = new BufferedReader(new InputStreamReader(socket. >> getInputStream())); >> >> It tells the C++ to quiesce and exit which apparently it does very >> quickly. It looks like >> >> writer.write("QUIESCEANDSTOP"); >> >> succeeded, but the code fails here >> >> BufferedReader in = new BufferedReader(new InputStreamReader(socket. >> getInputStream())); >> which means that the C++ socket is dead (process exited) >> >> It seems that the code should tolerate such condition and not "scream" >> Exception :) >> >> I think a try catch should be added around this code and on >> SocketException we should just log some informative >> message like "Remote process socket closed - It may have terminated" or >> something similar to that. >> >> So this does not seem like a critical error although this should be >> cleaned up. >> >> -jerry >> >> On Fri, Sep 23, 2016 at 9:41 AM, David Fox <david@humedica.com> >>wrote: >> >>> I was able to install that, and I’m not getting the >>>NullPointerException. >>> The example annotator still runs successfully, but now I get >>> >>> --- >>> >>> Attempting to deploy descriptors/Deploy_MeetingAnnotator.xml ... >>> Waiting
Re: NullPointerException when RunRemoteAsyncAE tries to unload (UIMA AS and UIMA C++)
I was able to install that, and I’m not getting the NullPointerException. The example annotator still runs successfully, but now I get --- Attempting to deploy descriptors/Deploy_MeetingAnnotator.xml ... Waiting for Uima C++ service to connect... Waiting for Uima C++ service to connect... Waiting for Uima C++ service to report init status... deployCppService.cpp91 Start receiving messages Startinging GetMetaData instance Starting Annotator instance 0 deployCppService.cpp UIMA C++ Service MeetingAnnotator at tcp://localhost:61616 Ready to process... Uima C++ service at MeetingAnnotator Ready to process... UIMA AS Service Initialization Complete Completed 8 documents; 17165 characters Time Elapsed : 3705 ms 0 8 apr_socket_recv command=SHUTDOWN deployCppService.cpp Shutdown started. Stopping GetMetaData instance Stopping Annotator instance 0 deployCppService.cpp Shutdown done. UimacppServiceManagement service reports shutdown DONE UimacppShutdownHook sending quiesce message java.net.SocketException: Socket is closed at java.net.Socket.getInputStream(Socket.java:903) at org.apache.uima.aae.controller.UimacppServiceManagement.quiesceAndStop(Uima cppServiceManagement.java:88) at org.apache.uima.aae.controller.UimacppServiceManagement.quiesceAndShutdown( UimacppServiceManagement.java:397) at org.apache.uima.aae.controller.UimacppShutdownHook.run(UimacppServiceContro ller.java:1105) UimacppShutdownHook destroy C++ process ― I’ve attached the uima.log On 9/22/16, 3:54 PM, "Jaroslaw Cwiklik" <uim...@gmail.com> wrote: >David, I've deployed uima-as 2.9.0-SNAPSHOT here: > >https://urldefense.proofpoint.com/v2/url?u=https-3A__repository.apache.org >_content_repositories_snapshots_org_apache_uima_uima-2Das_2.9.0-2DSNAPSHOT >_=DQIFaQ=3XrKki35ZWuh8X2qbeRISQ=BYS7q6K6Famz8NiMJzvOgYA-WQSvBt9z6TEb >aT3nnNM=NnQy2jfAk4Dyg-uijgdjHHbg6pxfdvwDkCn3166fVc8=y7GSTfzyEzhkYgFGy1 >vLLmDJ82il_hzMP7Y4LQjuBPg= > >there is a binary tarball (tar.gz) in there. > >This is a trunk level uima-as code for the upcoming 2.9.0 uima-as release. >It has not been thoroughly >tested or vetted yet. It has some fixes and improvements since 2.8.1. It >includes a new AMQ broker >version 5.14.0. It also has a fix for the problem you've reported. > >-jerry > > >On Thu, Sep 22, 2016 at 3:15 PM, Jaroslaw Cwiklik <uim...@gmail.com> >wrote: > >> Another possibility is to deploy a snapshot in a public maven repo. >> Hopefully you can get the binary tarball >> onto your machines somehow. I will to do that know and let you know >>where >> to get this. >> >> -jerry >> >> On Thu, Sep 22, 2016 at 3:01 PM, David Fox <david@humedica.com> >>wrote: >> >>> Building from svn on our linux machines is a bit tricky, because they >>>are >>> cut off from the internet by a firewall, and maven install wants >>>internet >>> access to download dependencies. It looks like there are workarounds >>>for >>> that >>> >>>(https://urldefense.proofpoint.com/v2/url?u=http-3A__stackoverflow.com_q >>>uestions_7233328_how-2Ddo-2Di-2Dconfigu=DQIFaQ=3XrKki35ZWuh8X2qbeRIS >>>Q=BYS7q6K6Famz8NiMJzvOgYA-WQSvBt9z6TEbaT3nnNM=NnQy2jfAk4Dyg-uijgdjHH >>>bg6pxfdvwDkCn3166fVc8=sk4NjHKmY1SCj4WAEH0A59C50xulbEthh1b5K5egbv0= >>> re-maven-for-of >>> fline-development >>> >>><https://urldefense.proofpoint.com/v2/url?u=http-3A__stackoverflow.com_q >>>uestions_7233328_how-2Ddo-2Di-2Dconfigure-2Dmaven-2Dfor-2Doffline-2Ddeve >>>lopment=DQIFaQ=3XrKki35ZWuh8X2qbeRISQ=BYS7q6K6Famz8NiMJzvOgYA-WQSv >>>Bt9z6TEbaT3nnNM=NnQy2jfAk4Dyg-uijgdjHHbg6pxfdvwDkCn3166fVc8=pMHCokJs >>>jTNk_XvWxddISyww6QBt65VZmeVGKLTLK94= > >>> suggests Artifactory), but it may take a day or two >>> before I can get everything set up. >>> >>> I could build it on my dev machine (OS X), but with the binary >>>install, I >>> wasn’t able to get the sample UIMA C++ code to build on Mac OS X. (My >>> eventual plan is to get linux on my dev machine via virtual box, but >>>that >>> means more software to install…) >>> >>> So anyway, I’ll try to get this working so I can build from svn before >>>the >>> fix, reproduce the problem, and then update and verify the fix. >>> >>> Thanks, >>> David >>> >>> On 9/22/16, 1:23 PM, "Jaroslaw Cwiklik" <uim...@gmail.com> wrote: >>> >>> >I think I see the bug. Do you know how to build uima-as from svn? >>> > >>> &
Re: NullPointerException when RunRemoteAsyncAE tries to unload (UIMA AS and UIMA C++)
Thanks for the quick response Yes, it occurs every time, at the same line in RunRemoteAsyncAE and org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_ impl.undeploy(BaseUIMAAsynchronousEngine_impl.java:964) On 9/22/16, 11:15 AM, "Jaroslaw Cwiklik" <uim...@gmail.com> wrote: >This warning > >WARN Transport - Transport Connection to: tcp:// >127.0.0.1:56261 failed: java.io.EOFException > >is from brokers log or console I suspect. It just means that the broker >lost a connection to a client. Looks like the client >went away without closing a connection. This is probably a side-effect of >NPE. > >Is the NPE repeatable? Does it happen each time you run this test? Looking >for a hint if there is a race >condition in the UIMA-AS code. > > > >On Thu, Sep 22, 2016 at 10:54 AM, David Fox <david@humedica.com> >wrote: > >> I¹m using the UIMA C++ framework 2.4.0 (32-bit) on linux (CentOS 6.5), >>and >> following the instructions >>https://urldefense.proofpoint.com/v2/url?u=https-3A__uima.apache.org_d_ >>=DQIFaQ=3XrKki35ZWuh8X2qbeRISQ=BYS7q6K6Famz8NiMJzvOgYA-WQSvBt9z6TEbaT >>3nnNM=dFZSJENJkyCDLeCQuBn89fBrTrsdKh4RIWwwPV7r0o8=CaIoLZ_7Je-TYsr0ZIP >>xs98mHRestzC4RtgVSp7kJaY= >> uimacpp-2.4.0/docs/overview_and_setup.html. I¹ve successfully built >> DaveDetector and run it with the standalone C++ pipeline (runAECpp) and >> within a Java pipeline via JNI. I¹ve now installed UIMA AS 2.8.1 so I >>can >> follow the instructions under 2.3 Testing Interoperability with UIMA AS >> >> >> cd $UIMACPP_HOME/examples/tutorial >> runRemoteAsyncAE.sh tcp://localhost:61616 MeetingAnnotator \ >> -d descriptors/Deploy_MeetingAnnotator.xml >> >> >> I get a NullPointerException and a transport warning about a >> java.io.EOFException >> >> >> runRemoteAsyncAE.sh tcp://localhost:61616 MeetingAnnotator -d >> descriptors/Deploy_MeetingAnnotator.xml -o `pwd`/out >> >> Attempting to deploy descriptors/Deploy_MeetingAnnotator.xml ... >> >> Waiting for Uima C++ service to connect... >> >> Waiting for Uima C++ service to connect... >> >> Waiting for Uima C++ service to report init status... >> >> deployCppService.cpp91 Start receiving messages >> >> >> Startinging GetMetaData instance >> >> >> Starting Annotator instance 0 >> >> >> deployCppService.cpp UIMA C++ Service MeetingAnnotator at >> tcp://localhost:61616 Ready to process... >> >> >> Uima C++ service at MeetingAnnotator Ready to process... >> >> UIMA AS Service Initialization Complete >> >> .Completed 1 documents >> >> Time Elapsed : 2184 ms >> >> java.lang.NullPointerException >> >> at >>org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_ >> impl.undeploy(BaseUIMAAsynchronousEngine_impl.java:964) >> >> at >>org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_ >> impl.undeploy(BaseUIMAAsynchronousEngine_impl.java:906) >> >> at org.apache.uima.examples.as.RunRemoteAsyncAE.run( >> RunRemoteAsyncAE.java:260) >> >> at org.apache.uima.examples.as.RunRemoteAsyncAE.main( >> RunRemoteAsyncAE.java:314) >> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> >> at sun.reflect.NativeMethodAccessorImpl.invoke( >> NativeMethodAccessorImpl.java:62) >> >> at sun.reflect.DelegatingMethodAccessorImpl.invoke( >> DelegatingMethodAccessorImpl.java:43) >> >> at java.lang.reflect.Method.invoke(Method.java:497) >> >> at org.apache.uima.bootstrap.UimaBootstrap.main( >> UimaBootstrap.java:87) >> >> WARN Transport - Transport Connection to: tcp:// >> 127.0.0.1:56221 failed: java.io.EOFException >> >> I'm >> I can also run with a collection reader: >> >> >> runRemoteAsyncAE.sh tcp://localhost:61616 MeetingAnnotator -d >> descriptors/Deploy_MeetingAnnotator.xml -o `pwd`/out -c >> /misc/NLP/dfox/scratch/uima/apache-uima-as-2.8.1/examples/ >> descriptors/collection_reader/FileSystemCollectionReader.mine.xml >> >> (where the modified collection reader points to >>$UIMA_HOME/examples/data) >> in which case I get ³Completed 8 documents², and xmi output for each in >>the >> out/ directory, but still get the same exception and warning. Based on >> that (plus looking at the RunRemoteAsyncAE java source), it looks like >>my >> analysis engine is running fine, but