[ANNOUNCE] Apache UIMA DUCC 2.1.0 released

[Jaroslaw Cwiklik]

The Apache UIMA team is pleased to announce the release of the UIMA DUCC,
version 2.1.0.

DUCC stands for Distributed UIMA Cluster Computing. DUCC is a cluster
management system providing tooling, management, and scheduling facilities
to automate the scale-out of applications written to the UIMA framework.
Core UIMA provides a generalized framework for applications that process
unstructured information such as human language, but does not provide a
scale-out mechanism. UIMA-AS provides a scale-out mechanism to distribute
UIMA pipelines over a cluster of computing resources, but does not provide
job or cluster management of the resources. DUCC defines a formal job model
that closely maps to a standard UIMA pipeline. Around this job model DUCC
provides cluster management services to automate the scale-out of UIMA
pipelines over computing clusters.

The main changes in this release are:

- DUCC framework now runs on Java 7 or Java 8
- Ubuntu and RHEL 7 support
- cgroup enhancements
 - uses standard cgroups organization
 - supports cgroup swappiness setting, restricting any swapping if
desired
- DUCC state and history storage moved from flat files to Cassandra DB,
reducing storage size 5x
- Ships with the latest UIMA-AS v2.8.1
- Ships with recent ActiveMQ v5.13.2
- DUCC's UIMA-AS services support failover and ssl connectors
- Many DUCC webpage improvements
- Clear user display of DUCC classes and relation to machines
- Robust handling of dynamic changes to DUCC class and nodepool definitions
- Full support of nodepools with different quantum
- DUCC broker access restricted to user ducc
- Eliminate need for user home directories located on a shared filesystem
- Built-in Job error handler programmable per job
- Migration utility for DUCC updates
- Change to vary-off behavior to facilitate cluster management
- Horizontal stacking of services instance allocations
- java-viaducc improvements including separation of stdout from stderr
respoonses
- An alert banner is displayed on ducc-mon pages if daemons are down
- Promoted DUCC from sandbox to the regular Apache project in the SVN

For a full list of the changes, please refer to Jira report:
http://uima.apache.org/d/uima-ducc-2.1.0/issuesFixed/jira-report.html

More information about DUCC can be found here:
http://uima.apache.org/doc-uimaducc-whatitam.html

-Jerry Cwiklik, for the Apache UIMA community

Re: CPE memory usage

[Richard Eckart de Castilho]

Do you have code for a minimal test case?

Cheers,

-- Richard

> On 08.08.2016, at 15:31,  
>  wrote:
> 
> Hi Richard!
> 
> I've changed the document reader to a kind of no-op-reader, that always sets 
> the document text to an empty string: same behavior, but much slower increase 
> in memory usage.
> 
> Cheers,
> Armin

Re: CPE memory usage

[Richard Eckart de Castilho]

Did you try using a different reader?

Cheers,

-- Richard

> On 08.08.2016, at 08:10,  
>  wrote:
> 
> Hi!
> 
> I'm using uimaFIT 2.2.0 and uimaj 2.8.1. The collectection processing engine 
> is slowy eating up all memory until it gets killed by the system. This 
> happens even when I'm just runnging a collection reader and no other 
> compoments (no analysis at all). Does anyone has experiented a similar 
> behavior or has any ideas?
> 
> Best,
> Armin

CPE memory usage

[Armin.Wegner]

Hi!

I'm using uimaFIT 2.2.0 and uimaj 2.8.1. The collectection processing engine is 
slowy eating up all memory until it gets killed by the system. This happens 
even when I'm just runnging a collection reader and no other compoments (no 
analysis at all). Does anyone has experiented a similar behavior or has any 
ideas?

Best,
Armin


pgpW9RfLKIuAI.pgp
Description: PGP signature