Hi Eddie, Thanks! That is very helpful!
Thanks, Yi-Wen On Mon, Feb 22, 2016 at 2:59 PM, Eddie Epstein <[email protected]> wrote: > Hi Yi-Wen, > > Here is an example console from a normal DUCC job, with comments > > 2016.02.19 10:33:34 Fri Received > 2016.02.19 10:33:34 Fri WaitingForDriver ... waiting for the Job > Driver to initialize successfully > 2016.02.19 10:33:52 Fri WaitingForResources ... waiting for resources to > start one or more Job Process (pipelines) > 2016.02.19 10:34:00 Fri Assigned > 2016.02.19 10:34:02 Fri Initializing ... one or more > JP > launched and doing initialization > 2016.02.19 10:35:23 Fri Running ... initialization > completed successfully, processing work items from JD > 2016.02.19 10:36:14 Fri Completing ... job processes are > terminating > 2016.02.19 10:36:31 Fri Completed ... all user processes > have terminated > 2016.02.19 10:36:31 Fri completion type: EndOfJob > 2016.02.19 10:36:31 Fri rationale: state manager detected normal completion > ... all work items processed > > Your console indicates that the job driver was terminated before > initialization completed. > There should be a lot more information in the job's log directory. Feel > free to zip up and email that data to Lou or I, as long as it does *not* > contain any confidential information. > > Regards, > Eddie > > > On Mon, Feb 22, 2016 at 5:18 PM, Yi-Wen Liu <[email protected]> wrote: > > > Hi, > > > > The monitor output from ducc.log: > > > > 2015.11.30 02:27:16 Mon Received > > 2015.11.30 02:27:16 Mon WaitingForDriver > > 2015.11.30 02:27:44 Mon WaitingForResources > > 2015.11.30 02:27:56 Mon Assigned > > 2015.11.30 02:28:04 Mon Initializing > > 2015.11.30 02:29:35 Mon Completing > > 2015.11.30 02:30:18 Mon Completed > > 2015.11.30 02:30:18 Mon completion type: CanceledByDriver > > 2015.11.30 02:30:18 Mon rationale: unspecified > > > > > > > > Thanks, > > Yi-Wen > > > > On Sun, Feb 21, 2016 at 1:45 PM, Eddie Epstein <[email protected]> > > wrote: > > > > > Hi, > > > Can you please paste the full monitor output from the DUCC job submit > > here? > > > Thanks, > > > Eddie > > > > > > On Sun, Feb 21, 2016 at 3:26 PM, Yi-Wen Liu <[email protected]> wrote: > > > > > > > Hi, > > > > > > > > Because when I used wait_for_completion, I saw the terminal print out > > the > > > > step by step state, and after it finished, the output file was not > > > > generated yet. The job status on web server page might still stay in > > > > "waiting for resources" instead of completion. > > > > > > > > And sorry for the confusion, I used "main process" to refer to the > > > process > > > > generating output (the second one below). > > > > > > > > Time axis----> > > > > *The terminal* status:waiting for driver -> ....-> > > > status: > > > > completed (point A, no output) > > > > *The web server page *initializing -> waiting for > > > > > > > > > > > > > > resouces............................................................->completion > > > > (point B, has output) > > > > > > > > And my question is when can I know the point when the output is > > generated > > > > (point B)? > > > > If I use wait_for_completion, it is telling me the point A but there > is > > > no > > > > output yet. > > > > > > > > If I misunderstand anything please correct me, thanks for your > > patience! > > > > > > > > Thanks, > > > > Yi-Wen > > > > > > > > On Sun, Feb 21, 2016 at 6:05 AM, Lou DeGenaro < > [email protected]> > > > > wrote: > > > > > > > > > Yi-Wen, > > > > > > > > > > When submitting a DUCC Job, wait for completion should prevent the > > > return > > > > > until all your work items have been processed. > > > > > > > > > > What exactly do you mean by "main process"? > > > > > > > > > > Lou. > > > > > > > > > > On Sat, Feb 20, 2016 at 10:30 PM, Yi-Wen Liu <[email protected]> > > wrote: > > > > > > > > > > > Hello, > > > > > > > > > > > > I am using ducc in a web app, and I need to know when does ducc > > > finish > > > > > its > > > > > > work, ie, generate the output file. > > > > > > I use the two properties in my job file: > > > > > > > > > > > > cancel_on_interrupt > > > > > > wait_for_completion > > > > > > > > > > > > However, when I submitted job file, it did show the status on > > > terminal, > > > > > but > > > > > > it finished earlier than the main process, I have to wait for few > > > > minutes > > > > > > for the output file to be generated. > > > > > > > > > > > > Is there any way to get notified when the main process finishes > > > without > > > > > > looking at the web server page? > > > > > > > > > > > > Any suggestion is appreciated! > > > > > > > > > > > > > > > > > > Thanks, > > > > > > Yi-Wen > > > > > > > > > > > > > > > > > > > > >
