UIMA-DUCC support

All, The DUCC segment of the Apache UIMA project is in need of hands-on help. Development has come to a halt and fixes, such as upgrading 3rd party dependencies, are at a standstill. Please volunteer to take on responsibility for DUCC support or it will have to be retired. Thanks! Lou DeGenaro

Re: UIMA DUCC slow processing

Have you examined various DUCC and system log files for possible clues? On Fri, Jun 12, 2020 at 1:16 PM Dr. Raja M. Suleman < raja.m.sulai...@gmail.com> wrote: > Hi, > Thank you for your reply and I'm sorry I couldn't get back to this > earlier. > > To get a better picture of the processing

Re: DUCC without shared file system

for $DUCC_HOME. On Tue, Sep 3, 2019 at 5:06 PM Lou DeGenaro wrote: > I opened Jira https://issues.apache.org/jira/browse/UIMA-6121 to track > this issue. > > > On Tue, Sep 3, 2019 at 1:51 PM Lou DeGenaro > wrote: > >> You not need do anything special to run DUCC without a sh

Re: DUCC without shared file system

I opened Jira https://issues.apache.org/jira/browse/UIMA-6121 to track this issue. On Tue, Sep 3, 2019 at 1:51 PM Lou DeGenaro wrote: > You not need do anything special to run DUCC without a shared filesystem. > Simply install it on a local filesystem. However, there is one

Re: DUCC without shared file system

You not need do anything special to run DUCC without a shared filesystem. Simply install it on a local filesystem. However, there is one caveat. If the user's (e.g. DUCC jobs) log directory is not in a shared filesystem, then DUCC-Mon will not have access and the contents won't be viewable. I'll

Re: Service with Pinger "NoClassDefFoundError"

Yes! What ever classes/jars you need to run your pinger should be included in the service_ping_classpath. Lou. On Wed, Jan 16, 2019 at 12:13 PM Florian wrote: > Hello, > > thank you for your previous answer regarding the pinger, which I could > resolve. > > I have another question regarding

Re: Register service with customized pinger

Hello Florian, There is an example of a custom DUCC service running here http://uima-ducc-demo.apache.org:42133/service.details.jsp?name=CUSTOM:localhost:7175 See the Registry tab and note the "user" specified entries. These correspond to the ducc_services --register parameters. Lou. On Mon,

Re: DUCC job Issue

Below is a simple python script that runs two jobs sequentially. Lou. = #!/usr/bin/env python import os ducc_home = '/user/ducc/ducc_runtime' cmd_submit = os.path.join(ducc_home,'bin/ducc_submit') jobA = 'examples/simple/1.job' jobB = 'examples/simple/1.dd.job' file_jobA =

Re: DUCC Login setup on webserver

http://uima-ducc-demo.apache.org:42133/doc/duccbook.html#x1-22600012.13 Lou. On Mon, Nov 27, 2017 at 11:05 PM, priyank sharma wrote: > Hi! > > How to enable login on ducc webserver? > > Thanks in advance > > -- > Thanks and Regards > Priyank Sharma > >

Re: Ducc Service Registration Error

pinger data is stale > 21 Nov 2017 14:33:27,226 INFO SM.ServiceManagerComponent - N/A T[69] > publish Publishing State, active job count = 1 > > Thanks and Regards > Priyank Sharma > > On Tuesday 21 November 2017 12:44 PM, priyank sharma wrote: > >> We are using DUCC's B

Re: Ducc Installation

DUCC's web server could do better about this temporary critical components alert at boot time. I opened Jira https://issues.apache.org/jira/browse/UIMA-5651?filter=-1 to address this. Lou. On Mon, Nov 20, 2017 at 4:25 AM, priyank sharma wrote: > Thanks it helped. >

Re: Ducc Service Registration Error

Are you trying to use DUCC's Broker for your service (not allowed, pw protected) or do you have your own Broker? Lou. On Tue, Nov 21, 2017 at 1:11 AM, priyank sharma wrote: > Thanks It helped. > > > Now my service shows the state *Waiting+Alert, *when i opened the

Re: DUCC's job goes into infintie loop

re my problem clear to you now??? > > Thanks and Regards > Priyank Sharma > > On Wednesday 15 November 2017 03:57 PM, Lou DeGenaro wrote: > >> Please note that we make a clear distinction between "services" and >> "jobs". Earlier e-mail from

Re: DUCC's job goes into infintie loop

re deployed on the machine which was > disconnect from the cluster. > > Thanks and Regards > Priyank Sharma > > On Tuesday 14 November 2017 04:08 PM, Lou DeGenaro wrote: > >> What do you mean by "server down", precisely? Since we have no logs to >> look at we can onl

Re: DUCC's job goes into infintie loop

t;> Eddie >>>> >>>> On Fri, Nov 10, 2017 at 12:11 AM, priyank sharma < >>>> priyank.sha...@orkash.com> >>>> wrote: >>>> >>>> There is nothing on the work item page and performance page on the web >>>> >>>

Re: DUCC's job goes into infintie loop

ot able to pick the data from the data > source and no UIMA aggregator is working for that batches. > > Are the issue because of the java heap space? We are giving 4gb ram to the > job-process. > > Attaching the Log file. > > Thanks and Regards > Priyank Sharma > > On

Re: DUCC's job goes into infintie loop

The first place to look is in your job's logs. Visit the ducc-mon jobs page ducchost:42133/jobs.jsp then click on the id of your job. Examine the logs by clicking on each log file name looking for any revealing information. Feel free to post non-confidential snippets here, or If you'd like to

Re: UIMA analysis from a database

Apache DUCC is simple to install and use. It is designed to run scaled-out (multi-threaded) UIMA pipelines as well as supporting services (Python, Java, what-have-you). It distributes work of multiple users fairly across large or small machine clusters, including fitting multiple jobs on the

Re: DUCC Web Server Service Visibility and Default Reservations

It should always be the case that if you successfully "ducc_services --register..." a service it should appear on the ducc-mon Services page. When you click on the Id for a Service you should see zero of more deployments, each Running instance corresponding to an O/S process. If you ever want to

Re: DUCC Security Model for Service Deployment

> > > I'm registering services and running ducc as user ducc. > > > -John > > > > From: Lou DeGenaro <lou.degen...@gmail.com> > Sent: Tuesday, July 18, 2017 12:19:21 PM > To: user@uima.apache.org > Subject: Re: DUCC Security Model for Service Deployment

Re: DUCC Security Model for Service Deployment

DUCC has its own password protected AMQ broker used for daemon communications. Are your services trying to use DUCC's broker? Lou. On Tue, Jul 18, 2017 at 1:02 PM, Osborne, John David (Campus) < ozb...@uab.edu> wrote: > I am having deploying services with DUCC using a single system, single >

Re: DUCC multi-valued parameter in job descriptor

:51:04 AM org.apache.uima.ducc.test.randomsleep.CR initialize(94) INFO: ** BB language: de Lou. On Fri, Jul 7, 2017 at 8:11 AM, Lou DeGenaro <lou.degen...@gmail.com> wrote: > --driver_descriptor_CR_overrides "key1=value1 key2=value2 key3=value3" > > Hope this helps.

Re: DUCC job automatically fails and gives Reason,or extraordinary status as cancelled by User | DUCC Version: 2.0.1

iver driver > env vars: 3 > 19 May 2017 11:20:34,104 INFO OR.ProcessAccounting - 104791 addProcess 0 > added > 19 May 2017 11:20:34,104 INFO OR.JobFactory - N/A specification user:mario > 19 May 2017 11:20:34,104 INFO OR.JobFactory - N/A specification > signature:null > 19 May 2017 11

Re: DUCC job automatically fails and gives Reason,or extraordinary status as cancelled by User | DUCC Version: 2.0.1

though activemq is > installed on the system. > > Thanks and Regards > Priyank Sharma > > On Thursday 18 May 2017 01:03 PM, Lou DeGenaro wrote: > >> Priyank, >> >> You must have specified --cancel_on_interrupt when you submitted you job. >> This requ

Re: DUCC job automatically fails and gives Reason,or extraordinary status as cancelled by User | DUCC Version: 2.0.1

_on_interrupt and the client submitting >> the >> job were stopped. >> >> Eddie >> >> On Tue, May 16, 2017 at 8:31 AM, Lou DeGenaro <lou.degen...@gmail.com> >> wrote: >> >> Dunno why the connection would be refused. Are the JD and JP on th

Re: DUCC job automatically fails and gives Reason,or extraordinary status as cancelled by User | DUCC Version: 2.0.1

gt; 14 May 2017 18:49:08,757 INFO ActionEnd - T[283] engage seqNo=3496 > remote=S144.3348.34 ended > in getNext > in getNext > 14 May 2017 18:49:08,792 INFO ActionGet - T[483] engage seqNo=3517 > remote=S144.2443.32 > 14 May 2017 18:49:08,874 INFO ActionGet - T[482] engage seqNo=3518 >

Re: DUCC job automatically fails and gives Reason,or extraordinary status as cancelled by User | DUCC Version: 2.0.1

Hello, There are two parts: JP (one or more) and JD (one). You have shown the log from a JP, which is trying to contact the JD for more work. Can you share the JD log? Also, you can find me on HipChat https://apache.hipchat.com/chat/room/3665278 in about an hour from now. Lou. On Tue, May

Re: DUCC Java API job info

< bauma...@stud.uni-frankfurt.de> wrote: > Thanks for the json url. An addition to the DuccJobMonitor would be great! > > -Daniel > > > On 19.04.2017 20:22, Lou DeGenaro wrote: > >> There is no CLI query that gives the desired results. One could write a >>

Re: DUCC Java API job info

There is no CLI query that gives the desired results. One could write a python script to visit http://uima-ducc-demo.apache.org:42133/ducc-servlet/json-format-aaData-jobs and parse the resulting json. Should there be a Jira to provide a CLI query capability equivalent of the Jobs page? Either a

Re: Support for UIMA Ducc

It is not unusual for the Cassandra DB to take a short while to boot. My experience has been usually less than a minute for a new install, but that depends on the machine's capabilities and load. Please have a look at the DB log which can be found in DUCC_HOME//cassandra-server/logs and look for

Re: DUCC 2.0.1: Job is not going to complete State

Does the job's JD log file have any exceptions or errors? Lou. On Fri, Mar 25, 2016 at 2:49 AM, reshu.agarwal wrote: > Hi, > > I am facing a problem in DUCC 2.0.1 i.e. job is not completing even after > CR hasNext() returned false. I had tested my job with >

Re: DUCC - Work Item Queue Time Management

strict under previous DUCC versions. Lou. On Wed, Jan 13, 2016 at 4:23 AM, reshu.agarwal <reshu.agar...@orkash.com> wrote: > > But, I did not need to do this in case of DUCC 1.1.0 and 1.0.0. > > Reshu. > Signature > > > On 01/12/2016 06:35 PM, Lou DeGenaro wrote: >

Re: DUCC - Work Item Queue Time Management

gt; > Signature On 01/11/2016 06:57 PM, Lou DeGenaro wrote: > >> Reshu, >> >> Am I to understand that you did this: >> >> "Alternatively, you can place your own names.txt file either in the >> current >> working directory or in your config directory

Re: DUCC - Work Item Queue Time Management

;> Dear Lou, >> >> Sorry for this delay. I have tried this in DUCC 2.0.1 version and job >> successfully run with this but shows "Job -1 submitted". But if I remove >> this additional flag, It still shows DriverProcessFailed. >> >> Reshu. >&

Re: DUCC - Work Item Queue Time Management

s which are in lib folder suppose if you are using mysql > database for getting data and the mysql classes jar is in lib folder > instead of test.jar. > > Hope it will clarify the situation. > > Reshu. > > > On 10/01/2015 06:46 PM, Lou DeGenaro wrote: > >> Reshu,

Re: DUCC - Work Item Queue Time Management

eeded to run my > Collection Reader. > > > On 10/01/2015 05:21 PM, Lou DeGenaro wrote: > >> Reshu, >> >> In DUCC 2.0.0 we've introduced the idea of classpath separation so that >> the >> user classpath is not contaminated by the DUCC classpath. Thus,

Re: DUCC - Work Item Queue Time Management

relevant information to solve the issue. So far I'm unable to imagine how you are able to create java.lang.reflect.InvocationTargetException. Lou. On Thu, Oct 1, 2015 at 8:06 AM, Lou DeGenaro <lou.degen...@gmail.com> wrote: > Reshu, > > Are you able to share your (non-co

Re: DUCC - Work Item Queue Time Management

he-uima-ducc-2.1.0-SNAPSHOT/lib/apache-commons/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/google-gson/*:/home/ducc/apache-uima-ducc-2.1.0-SNAPSHOT/lib/springframework/* > arg[26]: org.apache.uima.ducc.common.main.DuccService > / > Reshu > > > /1001 Command launching.../On

Re: DUCC - Work Item Queue Time Management

, Lou DeGenaro <lou.degen...@gmail.com> wrote: > Reshu, > > This is a bug in DUCC 2.0.0. See https://issues.apache.org/jira/browse/ > UIMA-4576?jql=project%20%3D%20UIMA. > > Presently, you would need download the current DUCC source and build a > new tarball to get th

Re: DUCC Services keep stopping

1. Visit host:port/services.jsp 2. Click on the Id of the service, which will bring you to the Details page 3. Click on the Log file link and look for problems Lou. On Mon, May 18, 2015 at 2:39 AM, Satya Nand Kanodia satya.kano...@orkash.com wrote: Hi, I am using DUCC service with UIMA-AS

Re: Job gets stuck for infinite time

or If a system running a job driver goes down , DUCC Head should allocate a new job driver. On 03/30/2015 06:02 PM, Lou DeGenaro wrote: I don't think forcing the JD node and the broker on the same host will help much if your network has issues since all parts of DUCC communicate thru the network

Re: Job gets stuck for infinite time

, which is running broker. On 03/24/2015 06:00 PM, Lou DeGenaro wrote: By serious I simply meant that if your network is having problems then applications that are dependent on the network will be having problems, DUCC being one such application. Granted, DUCC's Job Driver should not take NPE

Re: Job gets stuck for infinite time

with database etc etc. So upgrading to DUCC 2.0 is a solution or we can do some thing else ? what did you mean by more serious problems? On 03/23/2015 06:26 PM, Lou DeGenaro wrote: A Jira already exists for this issue https://issues.apache.org/ jira/browse/ UIMA-4225. It has been marked Won't Fix

Re: DUCC-unstable behaviour od ducc

1.1.0 is configured. Thanks for your help. :-) Reshu. On 12/08/2014 04:24 PM, Lou DeGenaro wrote: What is the unstable behavior of DUCC 1.1.0 when running it alone? All kinds of bad things can happen if you run 2 DUCCs on the same set of machines. I'm willing to help, but am not sure I

Re: DUCC-unstable behaviour od ducc

04:45 PM, Lou DeGenaro wrote: Each node has a DUCC Agent daemon that sends heartbeats. There was a bug discovered after the release of 1.1 whereby the share calculation is incorrect (a rounding up problem that you observe). The impact of this bug should be minimal. The bug has been fixed

Re: DUCC-1.1.0: Machines are going down very frequently

using default configuration: ducc.agent.node.metrics.publish.rate=3 ducc.rm.node.stability = 5 Reshu. Signature On 11/12/2014 05:03 PM, Lou DeGenaro wrote: What do you have defined in your ducc.properties for ducc.rm.node.stability and ducc.agent.node.metrics.publish.rate? The Web

Re: DUCC 1.1.0- How to Run two DUCC version on same machines with different user

have changed the broker port and ws port too but still faced a problem in starting the ducc1.1.0 version simultaneously. Reshu. On 11/17/2014 05:34 PM, Lou DeGenaro wrote: The broker port is specifiable in ducc.properties. The default is ducc.broker.port = 61617. Lou. On Mon, Nov 17

Re: DUCC stuck at WaitingForResources on an Amazon Linux

Simon, Congratulations! You found a bug in DUCC's Web Server. It was incorrectly rounding up when reporting the number of shares for a machine. This issue is addressed by Jira 4104 https://issues.apache.org/jira/browse/UIMA-4104. Lou. On Fri, Nov 14, 2014 at 7:49 AM, Jim Challenger

Re: UIMA DUCC - Excessive Initialize Failure

Log files may be informative. There are system and user log files. System log files are located in ducc_runtime/logs. User log files (which may be more helpful here) are located in --log_directory path that was specified on the submit command (and can also be seen by employing the WebServer).

Re: UIMA DUCC - Excessive Initialize Failure

14:23, Lou DeGenaro lou.degen...@gmail.com wrote: Log files may be informative. There are system and user log files. System log files are located in ducc_runtime/logs. User log files (which may be more helpful here) are located in --log_directory path that was specified on the submit

Re: UIMA DUCC - Excessive Initialize Failure

and run work as 'ducc' but not other userids ? Lou. On Mon, Nov 3, 2014 at 12:49 PM, james.d.ba...@gmail.com wrote: There doesn’t seem to be anything of relevance in the user logs. In fact, there doesn’t seem to be much in them at all! On 3 Nov 2014, at 14:43, Lou DeGenaro lou.degen...@gmail.com

Re: Scale out tuning for jobs

Amit, DUCC should use all available resources as configured by your ducc.classes and ducc.nodes files. Lou. On Thu, Oct 16, 2014 at 11:59 AM, Amit Gupta gupta.v.a...@gmail.com wrote: Thanks for the clarification Burn, So indeed there is no way to force a job to scale out to maximum

Re: is DUCC having UIMA AS remote Analysis Engine functionality for Deployment descripter?

There are several paradigms provided by DUCC: 1. Jobs For these you use ducc_submit to tell DUCC about your UIMA CR and AE. 2. Services For these you use ducc_services to register and deploy your UIMA AE. 3. Managed Reservations For these you use ducc_process_submit to tell DUCC about your

Re: Infinte initialization of a process even after restarting DUCC

DUCC release). Lou. On Wed, Jul 9, 2014 at 7:12 AM, reshu.agarwal reshu.agar...@orkash.com wrote: On 07/09/2014 03:25 PM, Lou DeGenaro wrote: Upon re-start (presuming you used the default warm start) all previous running jobs are marked as Completed. If the the job itself is Completed yet

Re: CanceledByDriver status in DUCC

...@orkash.com wrote: On 06/05/2014 06:30 PM, Lou DeGenaro wrote: How long does it take to process your longest work item through your Hi Lou, It was 3 Minutes that time. We increased this to 10 minutes now for testing as you told me to increase it. I have also restarted DUCC. Till now, I didn't face

Re: CanceledByDriver status in DUCC

DUCC does not have an automatic resubmit capability. Usually a high error count is indicative of a flawed job, and the default DUCC plug-in error handler cancels the job after 15 errors have occurred so as to not waste resources. Normally, one debugs the pipeline before scaling-out and the

Re: status Lost=1 in DUCC

, reshu.agarwal reshu.agar...@orkash.comwrote: On 03/27/2014 08:13 PM, Lou DeGenaro wrote: he data being sent are values rather than keys in your CAS? If so, this is not really a best practice for DUCC use. Hi Lou, This is not the problem of how I send the data. My document contains some

Re: status Lost=1 in DUCC

directory, for example jd.out.log, *JD*.log... Lou. On Thu, Mar 27, 2014 at 1:31 AM, reshu.agarwal reshu.agar...@orkash.comwrote: On 03/26/2014 10:06 PM, Lou DeGenaro wrote: Hi Reshu, re: your answers to 5 6 6a. Is the data that populates the CAS the name of a document or the document

Re: status Lost=1 in DUCC

share so I can try to reproduce here? Lou. On Wed, Mar 26, 2014 at 9:20 AM, reshu.agarwal reshu.agar...@orkash.comwrote: Hi Lou, On 03/26/2014 04:27 PM, Lou DeGenaro wrote: Hi Reshu, The good news is that DUCC is functional since 1.job works. So we need to find out why your particular

Re: status Lost=1 in DUCC

Some questions: 1. Are you able to run a simple job, such as 1.job from the examples directory successfully? 2. Where does the warning appear, in a log file in the job's log directory? Is there any other information related to that warning? 3. Are there any exceptions in any of the logs in the

Re: DUCC not leaving Initializing State

(); } mh.frameworkTrace(cid, mid, exit); } So, my progress bar was returning null, so the CasGenerator just skip it, keep total as null. I don't know how this goes through in the DUCC Flow Controller, but it looks like outside of AllInOne, it causes a hang From: Lou

Re: DUCC not leaving Initializing State

Neal, Did getNext() get called and did it return a filled-in CAS? I tried a simple test case with my CR returning null for getProgress() and the Job seemed to run just fine. However, one may be penalized by the scheduler if is had to guess how much work the Job comprises. Lou. On Fri, Nov 15,