Re: DUCC 1.1.0- Remain in Completing state.

2016-01-05 Thread Eddie Epstein 
Hi Reshu,

Each DUCC machine has an agent responsible for starting and killing
processes.
There was a bug ( https://issues.apache.org/jira/browse/UIMA-4194 ) where
the
agent failed to issue a kill -9 against "hung" JPs when a job was stopping.
The fix is in v2.0.

Regards,
Eddie


On Tue, Jan 5, 2016 at 12:54 AM, reshu.agarwal 
wrote:

> I forget to mention one thing i.e. After Killing the job, next job is
> unable to initialize and remain in " WaitingForDriver" state. I have also
> checked sm.log,or.log,pm.log e.t.c. but failed to find any thing. I have to
> restart my DUCC for running job again.
>
> Reshu.
>
>
> On 01/05/2016 11:14 AM, reshu.agarwal wrote:
>
>> Hi,
>>
>> I am using DUCC 1.1.0 version. I am facing a issue with my job i.e. it
>> remains in completing state even after initializing the stop process. My
>> job used two processes. And Job Driver logs:
>>
>> Jan 04, 2016 12:43:13 PM
>> org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngineCommon_impl
>> stop
>> INFO: Stopping Asynchronous Client.
>> Jan 04, 2016 12:43:13 PM
>> org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngineCommon_impl
>> stop
>> INFO: Asynchronous Client Has Stopped.
>> Jan 04, 2016 12:43:13 PM
>> org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngineCommon_impl$SharedConnection
>> destroy
>> INFO: UIMA AS Client Shared Connection Has Been Closed
>> Jan 04, 2016 12:43:13 PM
>> org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_impl stop
>> INFO: UIMA AS Client Undeployed All Containers
>>
>> One process logs:
>>
>> Jan 04, 2016 12:44:50 PM
>> org.apache.uima.adapter.jms.activemq.JmsInputChannel stopChannel
>> INFO: Stopping Service JMS Transport. Service: ducc.jd.queue.87494
>> ShutdownNow false
>> Jan 04, 2016 12:44:50 PM
>> org.apache.uima.adapter.jms.activemq.JmsInputChannel stopChannel
>> INFO: Controller: ducc.jd.queue.87494 Stopped Listener on Endpoint:
>> queue://ducc.jd.queue.87494 Selector: Selector:Command=2000 OR Command=2002.
>>
>> But, other process do not have any log of stopping the process.
>>
>> The case is of not completely undeploying all processes. I have to use
>> command to cancel the process: /ducc_install/bin$ ./ducc_cancel --id 87494
>> --dpid 4529.
>>
>> Some times it cancelled the process otherwise I have to use "kill -9"
>> command to kill the job forcefully.
>>
>> Kindly help.
>>
>> Thanks in advanced.
>>
>> Reshu.
>>
>>
>>
>

Re: DUCC - Work Item Queue Time Management

2016-01-05 Thread reshu.agarwal 

Dear Lou,

Sorry for this delay. I have tried this in DUCC 2.0.1 version and job 
successfully run with this but shows "Job -1 submitted". But if I remove 
this additional flag, It still shows DriverProcessFailed.


Reshu.
On 10/06/2015 01:56 AM, Lou DeGenaro wrote:

Reshu,

Have you tried ducc_submit with the additional flag:

   *--all_in_one local*

?

Lou.

On Mon, Oct 5, 2015 at 12:25 AM, reshu.agarwal 
wrote:


Lou,

My job identifies the CR from test.jar but it also uses the other 3rd
Party libraries which are in lib folder suppose if you are using mysql
database for getting data and the mysql classes jar is in lib folder
instead of test.jar.

Hope it will clarify the situation.

Reshu.


On 10/01/2015 06:46 PM, Lou DeGenaro wrote:


Reshu,

I have tried submitting jobs with the following problems:

1. user CP with missing UIMA jars
2. user CP with missing CR jar
3. user CP with CR xml file that specifies non-existent CR class
4. user CP with CR that throws NPE upon construction

In all cases an exception is logged that gives relevant information to
solve the issue.  So far I'm unable to imagine how you are able to create
java.lang.reflect.InvocationTargetException.

Lou.

On Thu, Oct 1, 2015 at 8:06 AM, Lou DeGenaro 
wrote:

Reshu,

Are you able to share your (non-confidential) Job specification and
corresponding jar files that demonstrate the problem?

Lou.

On Thu, Oct 1, 2015 at 7:54 AM, reshu.agarwal 
wrote:

Lou,

Yes, classpath specification contains all classes needed to run my
Collection Reader.


On 10/01/2015 05:21 PM, Lou DeGenaro wrote:

Reshu,

In DUCC 2.0.0 we've introduced the idea of classpath separation so that
the
user classpath is not contaminated by the DUCC classpath.  Thus, in the
JD
there are 2 classpaths, one used by DUCC itself ("DUCC-side") and the
other
used when running user code ("user-side").

When the JD is running on the DUCC-side it uses the classpath specified
in
job.classpath.properties.  User code (e.g. your Collection Reader) does
not
run under this path.

When the JD is running on the user-side, it uses the Java classloading
employing the classpath specified in your job specification.  If this
classpath is incomplete then needed classes will not be loadable.  So
everything needed to run your Collection Reader must be explicitly
specified in the Job specification's (user-side) classpath.

Does the user-side classpath (the one in your job specification)
contain
all classes needed to run your Collection Reader?

Lou.


On Thu, Oct 1, 2015 at 12:52 AM, reshu.agarwal <
reshu.agar...@orkash.com
wrote:

Dear Lou,


I have also tried by specifying the complete path of test.jar i.e.
/home/ducc/Uima_Test/test.jar. Yes, My job required the directories
and
jar
in UserClasspath. Others are the uima and uima as jars. But the
problem
is
these jar are not actually available in jd classpath.

Reshu.

Signature On 09/28/2015 05:51 PM, Lou DeGenaro wrote:

Reshu,


By my eye, the -classpath for the JD itself is correct, as your seems
to
exactly match mine.

With respect to the user specified code, your
ducc.deploy.UserClasspath
differs from mine, as is expected.  However, I notice in your
UserClasspath
the following: /home/ducc/Uima_Test/lib/*:test.jar:.  There is no
path
to
test.jar?  Also, does your Job really use the other directories &
jars
in
UserClasspath?

Lou.

On Mon, Sep 28, 2015 at 7:52 AM, reshu.agarwal <
reshu.agar...@orkash.com>
wrote:

The log is:/

1000 Command to exec: /usr/java/jdk1.7.0_71/jre/bin/java

arg[1]:
-DDUCC_HOME=/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT
arg[2]:



-Dducc.deploy.configuration=/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/resources/ducc.properties
arg[3]: -Dducc.agent.process.state.update.port=56622
arg[4]: -Dducc.process.log.dir=/home/ducc/ducc/logs/67/
arg[5]: -Dducc.process.log.basename=67-JD-S211
arg[6]: -Dducc.job.id=67
arg[7]:



-Dducc.deploy.configuration=/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/resources/ducc.properties
arg[8]: -Dducc.deploy.components=jd
arg[9]: -Dducc.job.id=67
arg[10]: -Xmx300M
arg[11]: -Dducc.deploy.JobId=67
arg[12]:



-Dducc.deploy.CollectionReaderXml=desc/collection_reader/DBCollectionReader
arg[13]:



-Dducc.deploy.UserClasspath=/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/examples/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/lib/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/apache-uima/apache-activemq/lib/optional/*:/home/ducc/Uima_Test/lib/*:test.jar:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/uima-ducc/user/*
arg[14]: -Dducc.deploy.WorkItemTimeout=5
arg[15]: -Dducc.deploy.JobDirectory=/home/ducc/ducc/logs/
arg[16]:
-Dducc.deploy.JpFlowController=org.apache.uima.ducc.FlowController

Re: DUCC - Work Item Queue Time Management

2016-01-05 Thread reshu.agarwal 

Lou,

The problem is the inability to resolve config Path. In initializing Job 
driver, a class is using a file names.txt from jar's config/names.txt 
but tried to find the file from /home/ducc/Uima_Test/config/names.txt. 
The case is coming in this version of DUCC and in Job driver 
initialization. As service has been created without issue even if using 
the same initialization method as well as adding *--all_in_one local* in 
props file also run job successfully.


Hope this will help.

Thanks in advanced.

Reshu.

On 01/06/2016 11:11 AM, reshu.agarwal wrote:

Dear Lou,

Sorry for this delay. I have tried this in DUCC 2.0.1 version and job 
successfully run with this but shows "Job -1 submitted". But if I 
remove this additional flag, It still shows DriverProcessFailed.


Reshu.
On 10/06/2015 01:56 AM, Lou DeGenaro wrote:

Reshu,

Have you tried ducc_submit with the additional flag:

   *--all_in_one local*

?

Lou.

On Mon, Oct 5, 2015 at 12:25 AM, reshu.agarwal 


wrote:


Lou,

My job identifies the CR from test.jar but it also uses the other 3rd
Party libraries which are in lib folder suppose if you are using mysql
database for getting data and the mysql classes jar is in lib folder
instead of test.jar.

Hope it will clarify the situation.

Reshu.


On 10/01/2015 06:46 PM, Lou DeGenaro wrote:


Reshu,

I have tried submitting jobs with the following problems:

1. user CP with missing UIMA jars
2. user CP with missing CR jar
3. user CP with CR xml file that specifies non-existent CR class
4. user CP with CR that throws NPE upon construction

In all cases an exception is logged that gives relevant information to
solve the issue.  So far I'm unable to imagine how you are able to 
create

java.lang.reflect.InvocationTargetException.

Lou.

On Thu, Oct 1, 2015 at 8:06 AM, Lou DeGenaro 
wrote:

Reshu,

Are you able to share your (non-confidential) Job specification and
corresponding jar files that demonstrate the problem?

Lou.

On Thu, Oct 1, 2015 at 7:54 AM, reshu.agarwal 


wrote:

Lou,

Yes, classpath specification contains all classes needed to run my
Collection Reader.


On 10/01/2015 05:21 PM, Lou DeGenaro wrote:

Reshu,
In DUCC 2.0.0 we've introduced the idea of classpath separation 
so that

the
user classpath is not contaminated by the DUCC classpath.  Thus, 
in the

JD
there are 2 classpaths, one used by DUCC itself ("DUCC-side") 
and the

other
used when running user code ("user-side").

When the JD is running on the DUCC-side it uses the classpath 
specified

in
job.classpath.properties.  User code (e.g. your Collection 
Reader) does

not
run under this path.

When the JD is running on the user-side, it uses the Java 
classloading
employing the classpath specified in your job specification.  If 
this
classpath is incomplete then needed classes will not be 
loadable.  So

everything needed to run your Collection Reader must be explicitly
specified in the Job specification's (user-side) classpath.

Does the user-side classpath (the one in your job specification)
contain
all classes needed to run your Collection Reader?

Lou.


On Thu, Oct 1, 2015 at 12:52 AM, reshu.agarwal <
reshu.agar...@orkash.com
wrote:

Dear Lou,


I have also tried by specifying the complete path of test.jar i.e.
/home/ducc/Uima_Test/test.jar. Yes, My job required the 
directories

and
jar
in UserClasspath. Others are the uima and uima as jars. But the
problem
is
these jar are not actually available in jd classpath.

Reshu.

Signature On 09/28/2015 05:51 PM, Lou DeGenaro wrote:

Reshu,

By my eye, the -classpath for the JD itself is correct, as 
your seems

to
exactly match mine.

With respect to the user specified code, your
ducc.deploy.UserClasspath
differs from mine, as is expected.  However, I notice in your
UserClasspath
the following: /home/ducc/Uima_Test/lib/*:test.jar:.  There is no
path
to
test.jar?  Also, does your Job really use the other directories &
jars
in
UserClasspath?

Lou.

On Mon, Sep 28, 2015 at 7:52 AM, reshu.agarwal <
reshu.agar...@orkash.com>
wrote:

The log is:/

1000 Command to exec: /usr/java/jdk1.7.0_71/jre/bin/java

arg[1]:
-DDUCC_HOME=/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT
arg[2]:



-Dducc.deploy.configuration=/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/resources/ducc.properties 


arg[3]: -Dducc.agent.process.state.update.port=56622
arg[4]: -Dducc.process.log.dir=/home/ducc/ducc/logs/67/
arg[5]: -Dducc.process.log.basename=67-JD-S211
arg[6]: -Dducc.job.id=67
arg[7]:



-Dducc.deploy.configuration=/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/resources/ducc.properties 


arg[8]: -Dducc.deploy.components=jd
arg[9]: -Dducc.job.id=67
arg[10]: -Xmx300M
arg[11]: -Dducc.deploy.JobId=67
arg[12]:



-Dducc.deploy.CollectionReaderXml=desc/collection_reader/DBCollectionReader 


arg[13]: