Re: DUCC job Issue
Below is a simple python script that runs two jobs sequentially. Lou. = #!/usr/bin/env python import os ducc_home = '/user/ducc/ducc_runtime' cmd_submit = os.path.join(ducc_home,'bin/ducc_submit') jobA = 'examples/simple/1.job' jobB = 'examples/simple/1.dd.job' file_jobA = os.path.join(ducc_home,jobA) file_jobB = os.path.join(ducc_home,jobB) run_jobA = cmd_submit+' -f '+file_jobA+' '+'--wait_for_completion' run_jobB = cmd_submit+' -f '+file_jobB+' '+'--wait_for_completion' print 'jobA: submit '+jobA result = os.system(run_jobA) print 'jobA: '+str(result) print 'jobB: submit '+jobB result = os.system(run_jobA) print 'jobB: '+str(result) On Fri, Apr 20, 2018 at 9:34 AM, Eddie Epstein wrote: > DUCC is designed for multi-user environments, and in particular tries to > balance resources fairly quickly across users in a fair-share allocation. > The default mechanism used is preemption. To eliminate preemption specify a > "non-preemptable" scheduling class for jobs such as "fixed". > > Other options that could be of interest include: > > ducc.rm.initialization.cap > This limits allocation to jobs until initialization is successful, limiting > the "damage" to other running preemptable jobs if a new job will not even > initialize. > > ducc.rm.expand.by.doubling > This limits the rate at which resources are allocated to allow gaining some > knowledge about job throughput to avoid over allocating resources. > > ducc.rm.prediction > Used along with doubling to avoid unnecessary allocation. > > Regards, > Eddie > > > On Fri, Apr 20, 2018 at 5:31 AM, priyank sharma > > wrote: > > > Hey! > > > > I am facing trouble while running one job at a time in ducc. I want that > > if one job is running the other one should wait for it to complete and > then > > start. > > My configuration file is attached below. > > Please help me with what I am missing. > > > > -- > > Thanks and Regards > > Priyank Sharma > > > > >
Re: DUCC job Issue
DUCC is designed for multi-user environments, and in particular tries to balance resources fairly quickly across users in a fair-share allocation. The default mechanism used is preemption. To eliminate preemption specify a "non-preemptable" scheduling class for jobs such as "fixed". Other options that could be of interest include: ducc.rm.initialization.cap This limits allocation to jobs until initialization is successful, limiting the "damage" to other running preemptable jobs if a new job will not even initialize. ducc.rm.expand.by.doubling This limits the rate at which resources are allocated to allow gaining some knowledge about job throughput to avoid over allocating resources. ducc.rm.prediction Used along with doubling to avoid unnecessary allocation. Regards, Eddie On Fri, Apr 20, 2018 at 5:31 AM, priyank sharma wrote: > Hey! > > I am facing trouble while running one job at a time in ducc. I want that > if one job is running the other one should wait for it to complete and then > start. > My configuration file is attached below. > Please help me with what I am missing. > > -- > Thanks and Regards > Priyank Sharma > >
DUCC job Issue
Hey! I am facing trouble while running one job at a time in ducc. I want that if one job is running the other one should wait for it to complete and then start. My configuration file is attached below. Please help me with what I am missing. -- Thanks and Regards Priyank Sharma working_directory /mario/Uima_newEnglishPipeline driver_descriptor_CRdesc/orkash/CollectionReader/AccumuloReaderDescriptor process_descriptor_AE desc/orkash/ae/aggregate/Testae3Descriptor process_descriptor_CC desc/orkash/CasConsumer/ElasticSearchCasConsumerDescriptor classpath UimaEnglish8.jar description UIMA_v3_Test--> 5acca75c416f23049230ae95 TO 5accadb4416f23049230b665 process_memory_size 6 process_per_item_time_max 10 process_deployments_max 4 process_thread_count1 process_failures_limit 50 process_jvm_args-Xms4g -XX:+UseConcMarkSweepGC #service_dependency UIMA-AS:SVOQueueNew:tcp://S145:61617 UIMA-AS:NLPSentiment:tcp://S145:61617 #scheduling_class np_serv
