Re: DUCC doesn't use all available machines

2014-11-30 7:25 GMT-06:00 Eddie Epstein : > On Sat, Nov 29, 2014 at 4:46 PM, Simon Hafner wrote: > >> I've thrown some numbers at it (doubling each) and it's running at >> comfortable 125 procs. However, at about 6.1k of 6.5k items, the procs >> drop down to 30.

Re: DUCC doesn't use all available machines

> > DUCC would have to be restarted for the JD size parameters to take effect. > > One of the current DUCC development items is to significantly reduce the > memory needed per work item, and raise the default limit for concurrent > work items by two or three orders of magnitude. &

Re: DUCC doesn't use all available machines

lable for all preemptable >> jobs. >> >> To see more JPs, increase the number and/or size of the input text files, >> or decrease the number of pipeline threads per JP. >> >> Note that it can be counter productive to run "too many" pipeline >> thr

Re: Ducc: Rename failed

2014-11-28 14:18 GMT-06:00 Eddie Epstein : > To debug, please add the following option to the job submission: > --all_in_one local > > This will run all the code in a single process on the machine doing the > submit. Hopefully the log file and/or console will be more informative. Yes, that helped.

Re: Ducc: Rename failed

2014-11-28 10:45 GMT-06:00 Eddie Epstein : > DuccCasCC component has presumably created > /home/ducc/analysis/txt.processed/5911.txt_0_processed.zip_temp and written > to it? I don't know, the _temp file doesn't exist anymore. > Did you run this sample job in something other than cluster mode? I g

Ducc: Rename failed

When running DUCC in cluster mode, I get "Rename failed". The file mentioned in the error message exists in the txt.processed/ directory. The mount is via nfs (rw,sync,insecure). org.apache.uima.resource.ResourceProcessException: Received Exception In Message From Service on Queue:ducc.jd.queue.75

Re: DUCC org.apache.uima.util.InvalidXMLException and no logs

2014-11-27 11:44 GMT-06:00 Eddie Epstein : > Those are the only two log files? Should be a ducc.log (probably with no > more info than on the console), and either one or both of the job driver > logfiles: jd.out.log and jobid-JD-jdnode-jdpid.log. If for some reason the > job driver failed to start,

DUCC org.apache.uima.util.InvalidXMLException and no logs

When launching the Raw Text example application, it doesn't load with the following error: [ducc@ip-10-0-0-164 analysis]$ MyAppDir=$PWD MyInputDir=$PWD/txt MyOutputDir=$PWD/txt.processed ~/ducc_install/bin/ducc_submit -f DuccRawTextSpec.job Job 50 submitted id:50 location:5991@ip-10-0-0-164 id:50

Re: DUCC stuck Waiting for Resources - new install on CentOS 6.5 VM

How many shares does your agent have available? 2014-11-18 14:37 GMT-06:00 Dan Heinze : > I've read the "DUCC stuck Waiting for Resources on Amazon..." thread. > I have a similar problem. I did my first install of DUCC yesterday on a > CentOS 6.5 VM with 9GB RAM. No problems with the install. ./

DUCC doesn't use all available machines

I fired the DuccRawTextSpec.job on a cluster consisting of three machines, with 100 documents. The scheduler only runs the processes on two machines instead of all three. Can I mess with a few config variables to make it use all three? id:22 state:Running total:100 done:0 error:0 retry:0 procs:1 i

Re: DUCC 1.1.0- How to Run two DUCC version on same machines with different user

2014-11-17 0:00 GMT-06:00 reshu.agarwal : > I want to run two DUCC version i.e. 1.0.0 and 1.1.0 on same machines with > different user. Can this be possible? Yes, that should be possible. You'll have to make sure there are no port conflicts, I'd guess the ActiveMQ port is hardcoded, the rest might

Re: DUCC stuck at WaitingForResources on an Amazon Linux

So to run effectively, I would need more memory, because the job wants two shares? ... Yes. With a larger node it works. What would be a reasonable memory size for a ducc node? 2014-11-14 9:38 GMT-06:00 Lou DeGenaro : > Simon, > > Congratulations! You found a bug in DUCC's Web Server. It was inc

Re: DUCC stuck at WaitingForResources on an Amazon Linux

re for reasons > the resources are not being allocated? > > Eddie > > On Wed, Nov 12, 2014 at 4:07 PM, Simon Hafner wrote: > >> 4 shares total, 2 in use. >> >> 2014-11-12 5:06 GMT-06:00 Lou DeGenaro : >> > Try looking at your DUCC's web server

Re: DUCC stuck at WaitingForResources on an Amazon Linux

4 shares total, 2 in use. 2014-11-12 5:06 GMT-06:00 Lou DeGenaro : > Try looking at your DUCC's web server. On the System -> Machines page > do you see any shares not inuse? > > Lou. > > On Wed, Nov 12, 2014 at 5:51 AM, Simon Hafner wrote: >> I'

DUCC stuck at WaitingForResources on an Amazon Linux

I've set up DUCC according to https://cwiki.apache.org/confluence/display/UIMA/DUCC ducc_install/bin/ducc_submit -f ducc_install/examples/simple/1.job the job is stuck at WaitingForResources. 12 Nov 2014 10:37:30,175 INFO Agent.LinuxNodeMetricsProcessor - process N/A ... Agent Collectin