Re: DUCC - Work Item Queue Time Management

Tue, 05 Jan 2016 22:33:07 -0800 

Lou,
The problem is the inability to resolve config Path. In initializing Job driver, a class is using a file names.txt from jar's config/names.txt but tried to find the file from /home/ducc/Uima_Test/config/names.txt. The case is coming in this version of DUCC and in Job driver initialization. As service has been created without issue even if using the same initialization method as well as adding *--all_in_one local* in props file also run job successfully. 

Hope this will help.

Thanks in advanced.

Reshu.

On 01/06/2016 11:11 AM, reshu.agarwal wrote:

Dear Lou,


Sorry for this delay. I have tried this in DUCC 2.0.1 version and job 
successfully run with this but shows "Job -1 submitted". But if I 
remove this additional flag, It still shows DriverProcessFailed.


Reshu.
On 10/06/2015 01:56 AM, Lou DeGenaro wrote:

Reshu,

Have you tried ducc_submit with the additional flag:

   *--all_in_one local*

?

Lou.


On Mon, Oct 5, 2015 at 12:25 AM, reshu.agarwal 
<reshu.agar...@orkash.com>

wrote:


Lou,

My job identifies the CR from test.jar but it also uses the other 3rd
Party libraries which are in lib folder suppose if you are using mysql
database for getting data and the mysql classes jar is in lib folder
instead of test.jar.

Hope it will clarify the situation.

Reshu.


On 10/01/2015 06:46 PM, Lou DeGenaro wrote:


Reshu,

I have tried submitting jobs with the following problems:

1. user CP with missing UIMA jars
2. user CP with missing CR jar
3. user CP with CR xml file that specifies non-existent CR class
4. user CP with CR that throws NPE upon construction

In all cases an exception is logged that gives relevant information to

solve the issue.  So far I'm unable to imagine how you are able to 
create

java.lang.reflect.InvocationTargetException.

Lou.

On Thu, Oct 1, 2015 at 8:06 AM, Lou DeGenaro <lou.degen...@gmail.com>
wrote:

Reshu,

Are you able to share your (non-confidential) Job specification and
corresponding jar files that demonstrate the problem?

Lou.


On Thu, Oct 1, 2015 at 7:54 AM, reshu.agarwal 
<reshu.agar...@orkash.com>

wrote:

Lou,

Yes, classpath specification contains all classes needed to run my
Collection Reader.


On 10/01/2015 05:21 PM, Lou DeGenaro wrote:

Reshu,

In DUCC 2.0.0 we've introduced the idea of classpath separation 
so that

the

user classpath is not contaminated by the DUCC classpath.  Thus, 
in the

JD

there are 2 classpaths, one used by DUCC itself ("DUCC-side") 
and the

other
used when running user code ("user-side").


When the JD is running on the DUCC-side it uses the classpath 
specified

in

job.classpath.properties.  User code (e.g. your Collection 
Reader) does

not
run under this path.


When the JD is running on the user-side, it uses the Java 
classloading
employing the classpath specified in your job specification.  If 
this
classpath is incomplete then needed classes will not be 
loadable.  So

everything needed to run your Collection Reader must be explicitly
specified in the Job specification's (user-side) classpath.

Does the user-side classpath (the one in your job specification)
contain
all classes needed to run your Collection Reader?

Lou.


On Thu, Oct 1, 2015 at 12:52 AM, reshu.agarwal <
reshu.agar...@orkash.com
wrote:

Dear Lou,


I have also tried by specifying the complete path of test.jar i.e.

/home/ducc/Uima_Test/test.jar. Yes, My job required the 
directories

and
jar
in UserClasspath. Others are the uima and uima as jars. But the
problem
is
these jar are not actually available in jd classpath.

Reshu.

Signature On 09/28/2015 05:51 PM, Lou DeGenaro wrote:

Reshu,


By my eye, the -classpath for the JD itself is correct, as 
your seems

to
exactly match mine.

With respect to the user specified code, your
ducc.deploy.UserClasspath
differs from mine, as is expected.  However, I notice in your
UserClasspath
the following: /home/ducc/Uima_Test/lib/*:test.jar:.  There is no
path
to
test.jar?  Also, does your Job really use the other directories &
jars
in
UserClasspath?

Lou.

On Mon, Sep 28, 2015 at 7:52 AM, reshu.agarwal <
reshu.agar...@orkash.com>
wrote:

The log is:/

1000 Command to exec: /usr/java/jdk1.7.0_71/jre/bin/java

        arg[1]:
-DDUCC_HOME=/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT
        arg[2]:




-Dducc.deploy.configuration=/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/resources/ducc.properties 


        arg[3]: -Dducc.agent.process.state.update.port=56622
        arg[4]: -Dducc.process.log.dir=/home/ducc/ducc/logs/67/
        arg[5]: -Dducc.process.log.basename=67-JD-S211
        arg[6]: -Dducc.job.id=67
        arg[7]:




-Dducc.deploy.configuration=/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/resources/ducc.properties 


        arg[8]: -Dducc.deploy.components=jd
        arg[9]: -Dducc.job.id=67
        arg[10]: -Xmx300M
        arg[11]: -Dducc.deploy.JobId=67
        arg[12]:




-Dducc.deploy.CollectionReaderXml=desc/collection_reader/DBCollectionReader 


        arg[13]:




-Dducc.deploy.UserClasspath=/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/examples/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/lib/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/optional/*:/home/ducc/Uima_Test/lib/*:test.jar:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/user/* 


        arg[14]: -Dducc.deploy.WorkItemTimeout=5

        arg[15]: 
-Dducc.deploy.JobDirectory=/home/ducc/ducc/logs/

        arg[16]:

-Dducc.deploy.JpFlowController=org.apache.uima.ducc.FlowController 


        arg[17]:
-Dducc.deploy.JpAeDescriptor=desc/ae/aggregate/AggDescriptor
        arg[18]:

-Dducc.deploy.JpCcDescriptor=desc/cas_consumer/CasConsumerDescriptor 


        arg[19]: -Dducc.deploy.JpDdName=DUCC.Job
        arg[20]: -Dducc.deploy.JpDdDescription=DUCC.Generated
        arg[21]: -Dducc.deploy.JpThreadCount=3
        arg[22]: -Dducc.deploy.JpDdBrokerURL=${broker.name}
        arg[23]: -Dducc.deploy.JpDdBrokerEndpoint=${queue.name}
        arg[24]: -classpath
        arg[25]:




/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/lib/uima-core.jar:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/apache-log4j/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/webserver/lib/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/http-client/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/apache-camel/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/apache-commons/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/google-gson/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/springframework/* 


        arg[26]: org.apache.uima.ducc.common.main.DuccService
/
Reshu


/1001 Command launching.../On 09/28/2015 05:11 PM, Lou DeGenaro
wrote:


I take it your are getting the previously posted stack track 
from

the

DUCC

Job's JD log file?  Near the top of that file should be 
something

like:

1000 Command to exec:
/home/degenaro/local/sun/jdk1.7.0_79/jre/bin/java
         arg[1]:



-DDUCC_HOME=/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT 


         arg[2]:




-Dducc.deploy.configuration=/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/resources/ducc.properties 


         arg[3]: -Dducc.agent.process.state.update.port=47941
         arg[4]:


-Dducc.process.log.dir=/tmp/ducc/driver/kiwi/ducc/logs/71370038/305/ 

         arg[5]: 
-Dducc.process.log.basename=305-JD-uima-ducc-demo-3

         arg[6]: -Dducc.job.id=305
         arg[7]:




-Dducc.deploy.configuration=/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/resources/ducc.properties 


         arg[8]: -Dducc.deploy.components=jd
         arg[9]: -Dducc.job.id=305
         arg[10]: -Xmx100M
         arg[11]: -Dducc.deploy.JobId=305
         arg[12]:




-Dducc.deploy.CollectionReaderXml=org.apache.uima.ducc.test.randomsleep.FixedSleepCR 


         arg[13]:




-Dducc.deploy.CollectionReaderCfg=jobfile=/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/examples/uima-ducc-vm/jobs/most.inputs 


compression=1 error_rate=0.0
         arg[14]:




-Dducc.deploy.UserClasspath=/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/examples/*:/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/lib/*:/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/user/* 


         arg[15]: -Dducc.deploy.WorkItemTimeout=12
         arg[16]:

-Dducc.deploy.JobDirectory=/tmp/ducc/driver/kiwi/ducc/logs/71370038 


         arg[17]:

-Dducc.deploy.JpFlowController=org.apache.uima.ducc.FlowController 


         arg[18]:




-Dducc.deploy.JpAeDescriptor=org.apache.uima.ducc.test.randomsleep.FixedSleepAE 


         arg[19]: -Dducc.deploy.JpDdName=DUCC.Job
         arg[20]: -Dducc.deploy.JpDdDescription=DUCC.Generated
         arg[21]: -Dducc.deploy.JpThreadCount=6
         arg[22]: -Dducc.deploy.JpDdBrokerURL=${broker.name}

         arg[23]: 
-Dducc.deploy.JpDdBrokerEndpoint=${queue.name}

         arg[24]: -classpath
         arg[25]:




/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/*:/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/lib/uima-core.jar:/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/lib/apache-log4j/*:/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/webserver/lib/*:/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/lib/http-client/*:/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/*:/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/lib/apache-camel/*:/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/lib/apache-commons/*:/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/lib/google-gson/*:/home/degenaro/ducc/versions/apache-uima-ducc-2.1.0-SNAPSHOT/lib/springframework/* 


         arg[26]: org.apache.uima.ducc.common.main.DuccService
1001 Command launching...


Do the -Dducc.deploy.UserClasspath and -classsapth look 
right in

yours?  Can you post yours so we can compare and contrast?

Lou.



On Mon, Sep 28, 2015 at 7:26 AM, reshu.agarwal <
reshu.agar...@orkash.com>
wrote:

My CR is in test.jar and third party jars are in


/home/ducc/Uima_test/lib/*. It correctly specified the 
location of

CR


otherwise It will throw "class not find" exception but It 
showed

error
in
initialization of third party class.


1.job run perfectly as well as the same specification of 
classpath

worked
for creating DUCC service for the same project.

If I considered the path is somehow incorrect then it will not
work
even

if I defined the same in jobclasspath.properties. I know I 
should

not
touch
it.

Thanks in advance.

Reshu.
Signature

**


On 09/25/2015 05:52 PM, Lou DeGenaro wrote:

Reshu,


Again, you should not be touching jobclasspath.properties.  
Your



opportunity to specify classpath is in your DUCC Job 
submission

itself
via
the "classpath" keyword.


The exception you posted shows the Job Driver (JD) is 
attempting

to
create

an instance of your Collection Reader (CR) based on the 
classpath

specified

in your submitted DUCC Job, but is unable to do so.  I 
suspect

the
classpath

in your DUCC Job is wrong or the jar files needed are 
somehow not

available
during runtime?

I presume that your CR is expected to be somewhere in

           /home/ducc/Uima_test/lib/*:
                  test.jar


Does this correctly specify the location of your DUCC 
Job's CR?

(Do
you
have extraneous white space in your DUCC Job's specified
classpath?)

As a sanity check are you able to run, for example, 1.job?

degenaro@uima-ducc-vm:~/ducc/ducc_runtime/examples/simple$
ducc_submit
--specification 1.job --wait_for_completion --timestamp
Job 85 submitted
25/09/2015 12:03:29 id:85 location:29496@uima-ducc-vm
25/09/2015 12:03:39 id:85 state:WaitingForDriver
25/09/2015 12:03:59 id:85 state:WaitingForResources
25/09/2015 12:04:09 id:85 state:Initializing

25/09/2015 12:04:30 id:85 state:Running total:15 done:6 
error:0

retry:0
procs:1

25/09/2015 12:04:40 id:85 state:Running total:15 done:11 
error:0

retry:0
procs:1

25/09/2015 12:04:50 id:85 state:Running total:15 done:14 
error:0

retry:0
procs:1
25/09/2015 12:05:00 id:85 state:Completing total:15 done:15
error:0
retry:0
procs:1
25/09/2015 12:05:10 id:85 state:Completed total:15 done:15
error:0
retry:0
procs:0

25/09/2015 12:05:10 id:85 rationale:state manager detected 
normal

completion
25/09/2015 12:05:10 id:85 rc:0


Lou.

On Fri, Sep 25, 2015 at 12:49 AM, reshu.agarwal <
reshu.agar...@orkash.com
wrote:

Lewis & Lou,


When I classified the required library in classpath like 
below,

Job

was

unsuccessful and Status is "DriverProcessFailed".

classpath



/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/examples/*: 

/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/lib/*: 









/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/*: 









/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/optional/*: 


/home/ducc/Uima_test/lib/*:
                  test.jar

As It said "Driver Process Failed" and JD's log file showed
error
about

not finding the classpath in job driver, I just tried to 
add my

library
in
jobclasspath.properties to be sure of problem.

25 Sep 2015 10:03:27,688  INFO JobDriverComponent - T[1]
verifySystemProperties ducc.deploy.WorkItemTimeout=5

25 Sep 2015 10:03:27,716  INFO JobDriverStateExchanger - 
T[1]

initializeTarget  http://S211:19988/or

25 Sep 2015 10:03:27,725  INFO JobDriver - T[1] 
advanceJdState

current=Prelaunch request=Initializing result=Initializing
25 Sep 2015 10:03:32,158 ERROR ProxyLogger - T[1]
loggifyUserException
java.lang.reflect.InvocationTargetException
              at

sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 


Method)
              at






sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) 


              at






sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 


              at

java.lang.reflect.Constructor.newInstance(Constructor.java:526) 


              at






org.apache.uima.ducc.container.jd.classload.ProxyJobDriverCollectionReader.prepare(ProxyJobDriverCollectionReader.java:164) 


              at






org.apache.uima.ducc.container.jd.classload.ProxyJobDriverCollectionReader.construct(ProxyJobDriverCollectionReader.java:135) 


              at






org.apache.uima.ducc.container.jd.classload.ProxyJobDriverCollectionReader.initialize(ProxyJobDriverCollectionReader.java:86) 


              at






org.apache.uima.ducc.container.jd.classload.ProxyJobDriverCollectionReader.<init>(ProxyJobDriverCollectionReader.java:72) 


              at






org.apache.uima.ducc.container.jd.cas.CasManager.initialize(CasManager.java:51) 


              at






org.apache.uima.ducc.container.jd.cas.CasManager.<init>(CasManager.java:45) 


              at






org.apache.uima.ducc.container.jd.JobDriver.initialize(JobDriver.java:113) 


              at



org.apache.uima.ducc.container.jd.JobDriver.<init>(JobDriver.java:96) 


              at






org.apache.uima.ducc.container.jd.JobDriver.getInstance(JobDriver.java:61) 


              at






org.apache.uima.ducc.transport.configuration.jd.JobDriverComponent.createInstance(JobDriverComponent.java:86) 


              at






org.apache.uima.ducc.transport.configuration.jd.JobDriverComponent.<init>(JobDriverComponent.java:56) 


              at






org.apache.uima.ducc.transport.configuration.jd.JobDriverConfiguration.jobDriver(JobDriverConfiguration.java:180) 


              at






org.apache.uima.ducc.transport.configuration.jd.JobDriverConfiguration$$EnhancerByCGLIB$$309bc85e.CGLIB$jobDriver$1(<generated>) 


              at






org.apache.uima.ducc.transport.configuration.jd.JobDriverConfiguration$$EnhancerByCGLIB$$309bc85e$$FastClassByCGLIB$$699d0096.invoke(<generated>) 


              at

net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:215) 


              at






org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:210) 


              at






org.apache.uima.ducc.transport.configuration.jd.JobDriverConfiguration$$EnhancerByCGLIB$$309bc85e.jobDriver(<generated>) 


              at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
              at






sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 


              at






sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 

              at 
java.lang.reflect.Method.invoke(Method.java:606)

              at






org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:145) 


              at






org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:570) 


              at






org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:982) 


              at






org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:878) 


              at






org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:484) 


              at






org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:455) 


              at






org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293) 


              at






org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) 


              at






org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290) 


              at






org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:192) 


              at






org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:585) 


              at






org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:895) 


              at






org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:425) 


              at






org.springframework.context.annotation.AnnotationConfigApplicationContext.<init>(AnnotationConfigApplicationContext.java:65) 


              at




org.apache.uima.ducc.common.main.DuccService.boot(DuccService.java:160) 


              at




org.apache.uima.ducc.common.main.DuccService.main(DuccService.java:289) 





Hope this will clear my problem.

Thanks in advance.

Reshu



On 09/24/2015 06:28 PM, Burn Lewis wrote:


For DUCC 2.x the jobclasspath.properties file defines the 
JD &

JP


classpaths for JUST the ducc code in the JD & JP.  The 
user code

in

the

JD
(your collection reader) and in the JP (your annotator
pipeline)
uses
ONLY
the classpath you provide plus one ducc jar.


Adding UIMA and application jars to the 
jobclasspath.properties

file
should
not help your user code (it does help in 1.x which uses a
combined
ducc+user classpath.)

So the major change for DUCC 2.0 is that you must specify a
complete
classpath for your application.

~Burn

On Thu, Sep 24, 2015 at 7:59 AM, Lou DeGenaro <
lou.degen...@gmail.com
wrote:

Reshu,

Absent some extraordinary circumstance, you should not be
touching

jobclasspath.properties file.


Specify your classpath requirement using --classpath 
when you

submit
your

job or register your service.  This is where you'd add 
UIMA

jars,
for
example.

Lou.

On Tue, Sep 22, 2015 at 12:38 AM, reshu.agarwal <
reshu.agar...@orkash.com>
wrote:

Hi,


Thanks for replying. I have downloaded the latest code 
from

github

and


build it. Now, the problem of "Missing the
-Dducc.deploy.JdURL
property"
was resolved.

*Lewis:* I used the provided one
resources/jobdriver.classpath's
file
of
DUCC 2.0.0 and have to do the same in DUCC 2.1.0.'s
resources/jobclasspath.properties file.


I added the required lib folder for my job to this 
file. Now

the
file
be
like:

ducc.jobdriver.classpath = \
${DUCC_HOME}/lib/uima-ducc/*:\
${DUCC_HOME}/apache-uima/lib/uima-core.jar:\
${DUCC_HOME}/lib/apache-log4j/*:\
${DUCC_HOME}/webserver/lib/*:\
${DUCC_HOME}/lib/http-client/*:\
${DUCC_HOME}/apache-uima/apache-activemq/lib/*:\
${DUCC_HOME}/lib/apache-camel/*:\
${DUCC_HOME}/lib/apache-commons/*:\
${DUCC_HOME}/lib/google-gson/*:\
${DUCC_HOME}/lib/springframework/*:/

___/home/ducc/Uima_pipeline/lib/*<---------(I changed 
here

for
my
job.)_

ducc.jobprocess.classpath = \
${DUCC_HOME}/lib/uima-ducc/*:\
${DUCC_HOME}/apache-uima/lib/uima-core.jar:\
${DUCC_HOME}/lib/apache-log4j/*:\
${DUCC_HOME}/webserver/lib/*:\
${DUCC_HOME}/lib/http-client/*:\
${DUCC_HOME}/apache-uima/apache-activemq/lib/*:\

${DUCC_HOME}/apache-uima/apache-activemq/lib/optional/*:\
${DUCC_HOME}/lib/apache-camel/*:\
${DUCC_HOME}/lib/apache-commons/*:\
${DUCC_HOME}/lib/springframework/*


This changes works in DUCC 2.1.0 version and my job 
completed
successfully. But this is not a solution as these all 
jars

will
add
to

each

job even if not necessary. This lib folder contains third
party

jar

as

well

as UIMA and UIMA AS jars.

On 09/22/2015 01:56 AM, Burn Lewis wrote:

re your original problem of a missing UIMA class:

It should not be necessary to modify
resources/jobdriver.classpath

...


were

you using the one provided with 2.0 or do you have a 
locally

modified

one?


Please let us know what changes to the 2.0 one you 
had to

make.


You should just add the required UIMA jars to the 
classpath

you

provide


when you submit the job.  If you provide a deployment
descriptor
you'll
need to supply all the UIMA-AS jars, e.g.









${DUCC_HOME}/apache-uima/lib/*:${DUCC_HOME}/apache-uima/apache-activemq/lib/*:${DUCC_HOME}/apache-uima/apache-activemq/lib/optional/* 



otherwise you probably need only 1 jar, e.g.

*${DUCC_HOME}/apache-uima/lib/uima-core.jar*


Note that in these examples I've used the UIMA jars 
that are

included

with


DUCC, but in general it would be better if you used 
your own


copy of

UIMA,
at whatever level is best for your application.


In DUCC 1.x the DUCC jars and their dependencies were 
added

to
the

user's


classpath, but this often caused problems when DUCC 
code and

user

code

used


different versions of a 3rd party jar, so in DUCC 2.0 
we use

a


different

classloader for DUCC & user code, and add only one 
DUCC jar

to
the

user's

classpath.

~Burn

On Mon, Sep 21, 2015 at 9:18 AM, Jaroslaw Cwiklik <


cwik...@us.ibm.com>
wrote:


Reshu, if you have maven and svn installed on your 
machine

you
can

checkout the latest code from the svn:

svn co



https://svn.apache.org/repos/asf/uima/sandbox/uima-ducc/trunk/ 



.

and  build it with: mvn clean install

You'll get a new ducc tarball in target dir

         Jerry Cwiklik


IBM Watson RTP North Carolina

UIMA Extensions
4205 S MIAMI BLVD
DURHAM , NC , 27703-9141
United States
Building: 502  |  Floor: 02 |  Office: M210
Tel: 919-254-6641  TL:444-6641
Email: cwik...@us.ibm.com

[image: Inactive hide details for Lou DeGenaro
---09/21/2015
08:44:06
AM---Reshu, This is a bug in DUCC 2.0.0. See
https://issues.apac
]Lou

DeGenaro ---09/21/2015 08:44:06 AM---Reshu, This is 
a bug

in
DUCC

2.0.0.

See https://issues.apache.org/jira/browse/UIMA

From: Lou DeGenaro <lou.degen...@gmail.com>

To: user@uima.apache.org


Date: 09/21/2015 08:44 AM

Subject: Re: DUCC - Work Item Queue Time Management
------------------------------



Reshu,

This is a bug in DUCC 2.0.0. See
https://issues.apache.org/jira/browse/UIMA
-4576?jql=project%20%3D%20UIMA.


Presently, you would need download the current DUCC 
source

and
build a
new
tarball to get the fix.

In the mean time, I'll investigate how interim DUCC
releases
(tarballs)
are
posted to the Apache website.

Lou.

On Mon, Sep 21, 2015 at 7:25 AM, reshu.agarwal <

reshu.agar...@orkash.com

wrote:

Hi,

As you said:    "In DUCC 2.0 you must explicitly 
supply UIMA

in


the

classpath of your submission. This was not the case 
in DUCC

1.x

where

UIMA

was added by DUCC under the covers."

I defined the same but still facing the error. In JD


initialization,


I

defined the java class library path required in 
classpath

parameter
in

job


specification. But it was showing error until I 
added the

same
in

resources/jobdriver.classpath. After this It was


initialized and

then


started showing error of "Missing the 
-Dducc.deploy.JdURL

property".

I was getting java.lang.RuntimeException: Missing the
-Dducc.deploy.JdURL
property even in 1.job. Why this error is coming?

Thanks in Advance.

Reshu.


On 09/18/2015 02:47 PM, Lou DeGenaro wrote:

Reshu,

In DUCC 2.0 you must explicitly supply UIMA in the
classpath of

your


submission.  This was not the case in DUCC 1.x 
where UIMA

was
added

by

DUCC

under the covers.

In fact this gives you more flexibility in that you 
are no

loner

tied

to

using a particular version of UIMA.

Lou.

On Fri, Sep 18, 2015 at 12:24 AM, reshu.agarwal <

reshu.agar...@orkash.com

wrote:

Jerry,


I have tried DUCC 2.0.0 to run same job on it. I don't
know
why

but

same


job descriptor didn't work. It showed some 
exception at



initialization


time

which was not in case of 1.1.0.


Is there any changes regarding job descriptor or 
service

descriptor?

The


both did not work in my case for DUCC 2.0.0 but 
for DUCC

1.0.0

and


DUCC

1.1.0.

In Service descriptor it shows some spring Framework's

class


not

found

exception. See below:

*java.lang.NoClassDefFoundError:

org/springframework/context/ApplicationListener*

Thanks in advance.


Reshu.

On 09/17/2015 08:15 PM, Jaroslaw Cwiklik wrote:

Hi, can you try Ducc 2.0.0? It was recently released
into
Apache.

One

of

the key changes was to remove queues as means of

transport


between JD

(Job


Driver) and JP (Job Process). Instead, each JP uses 
HTTP to

request a

Work

Item (CAS) from a JD.

DUCC 1.1.0 has a concept of a WI timeout which I 
think

is 24


hours


by

default. A timer is started in a JD when each WI is

dispatched

to


a

JP.

If

the WI does not come back for whatever reason, the timer

pops


and

a

JD

will


attempt to retry that WI.

To debug your problem with DUCC 1.1.0 I suggest
attaching
JMX

console

to a


running JP to see where its threads are. Before 
doing

this,

check

JP


logs


to see if there is an exception.

Jerry

On Thu, Sep 17, 2015 at 4:32 AM, reshu.agarwal <


reshu.agar...@orkash.com>

wrote:

My DUCC version is 1.1.0.

On 09/17/2015 11:35 AM, reshu.agarwal wrote:

Hi,


I am facing a problem in DUCC that some 
documents were

shown

in

queue

but


did not get processed. In Job, work item list 
shows a


particular

work

item's status "queued" and queueing time is "4115
seconds".


I want to set queueing time of work item not more 
then 1

minute.


What

is


the reason for the same? Is there any method to 
solve

this?

How


can

I

set

maximum queueing time for work item?

Thanks in advance.

Reshu.









































					Reply via email to