Hi Yi-Wen, Here is an example console from a normal DUCC job, with comments
2016.02.19 10:33:34 Fri Received 2016.02.19 10:33:34 Fri WaitingForDriver ... waiting for the Job Driver to initialize successfully 2016.02.19 10:33:52 Fri WaitingForResources ... waiting for resources to start one or more Job Process (pipelines) 2016.02.19 10:34:00 Fri Assigned 2016.02.19 10:34:02 Fri Initializing ... one or more JP launched and doing initialization 2016.02.19 10:35:23 Fri Running ... initialization completed successfully, processing work items from JD 2016.02.19 10:36:14 Fri Completing ... job processes are terminating 2016.02.19 10:36:31 Fri Completed ... all user processes have terminated 2016.02.19 10:36:31 Fri completion type: EndOfJob 2016.02.19 10:36:31 Fri rationale: state manager detected normal completion ... all work items processed Your console indicates that the job driver was terminated before initialization completed. There should be a lot more information in the job's log directory. Feel free to zip up and email that data to Lou or I, as long as it does *not* contain any confidential information. Regards, Eddie On Mon, Feb 22, 2016 at 5:18 PM, Yi-Wen Liu <[email protected]> wrote: > Hi, > > The monitor output from ducc.log: > > 2015.11.30 02:27:16 Mon Received > 2015.11.30 02:27:16 Mon WaitingForDriver > 2015.11.30 02:27:44 Mon WaitingForResources > 2015.11.30 02:27:56 Mon Assigned > 2015.11.30 02:28:04 Mon Initializing > 2015.11.30 02:29:35 Mon Completing > 2015.11.30 02:30:18 Mon Completed > 2015.11.30 02:30:18 Mon completion type: CanceledByDriver > 2015.11.30 02:30:18 Mon rationale: unspecified > > > > Thanks, > Yi-Wen > > On Sun, Feb 21, 2016 at 1:45 PM, Eddie Epstein <[email protected]> > wrote: > > > Hi, > > Can you please paste the full monitor output from the DUCC job submit > here? > > Thanks, > > Eddie > > > > On Sun, Feb 21, 2016 at 3:26 PM, Yi-Wen Liu <[email protected]> wrote: > > > > > Hi, > > > > > > Because when I used wait_for_completion, I saw the terminal print out > the > > > step by step state, and after it finished, the output file was not > > > generated yet. The job status on web server page might still stay in > > > "waiting for resources" instead of completion. > > > > > > And sorry for the confusion, I used "main process" to refer to the > > process > > > generating output (the second one below). > > > > > > Time axis----> > > > *The terminal* status:waiting for driver -> ....-> > > status: > > > completed (point A, no output) > > > *The web server page *initializing -> waiting for > > > > > > > > > resouces............................................................->completion > > > (point B, has output) > > > > > > And my question is when can I know the point when the output is > generated > > > (point B)? > > > If I use wait_for_completion, it is telling me the point A but there is > > no > > > output yet. > > > > > > If I misunderstand anything please correct me, thanks for your > patience! > > > > > > Thanks, > > > Yi-Wen > > > > > > On Sun, Feb 21, 2016 at 6:05 AM, Lou DeGenaro <[email protected]> > > > wrote: > > > > > > > Yi-Wen, > > > > > > > > When submitting a DUCC Job, wait for completion should prevent the > > return > > > > until all your work items have been processed. > > > > > > > > What exactly do you mean by "main process"? > > > > > > > > Lou. > > > > > > > > On Sat, Feb 20, 2016 at 10:30 PM, Yi-Wen Liu <[email protected]> > wrote: > > > > > > > > > Hello, > > > > > > > > > > I am using ducc in a web app, and I need to know when does ducc > > finish > > > > its > > > > > work, ie, generate the output file. > > > > > I use the two properties in my job file: > > > > > > > > > > cancel_on_interrupt > > > > > wait_for_completion > > > > > > > > > > However, when I submitted job file, it did show the status on > > terminal, > > > > but > > > > > it finished earlier than the main process, I have to wait for few > > > minutes > > > > > for the output file to be generated. > > > > > > > > > > Is there any way to get notified when the main process finishes > > without > > > > > looking at the web server page? > > > > > > > > > > Any suggestion is appreciated! > > > > > > > > > > > > > > > Thanks, > > > > > Yi-Wen > > > > > > > > > > > > > > >
