Re: Cancel hung jobs
Thanks for the info. I finally ended up restarting the management servers since there was a DB split-brain issue underneath. FG On 2015-01-06 12:32 PM, ilya musayev wrote: Here are my notes on cleaning up stuck VmWorkJobQueue, i use MySQL workbench to make changes interactively... Messages in log file: (1013,VmWorkJobQueue, 2375) is reaching concurrency limit 1 (1013,VmWorkJobQueue, 2375) is reaching concurrency limit 1 (1433,VmWorkJobQueue, 2470) is reaching concurrency limit 1 (1434,VmWorkJobQueue, 2471) is reaching concurrency limit 1 (1475,VmWorkJobQueue, 2483) is reaching concurrency limit 1 (1477,VmWorkJobQueue, 2484) is reaching concurrency limit 1 (1478,VmWorkJobQueue, 596) is reaching concurrency limit 1 (981,VmWorkJobQueue, 2371) is reaching concurrency limit 1 -- DELETE ALL FROM BELOW - not exact mysql query, i just did select first and then deleted rows via mysql workbench. SELECT * FROM cloud.sync_queue_item where queue_id=981 or queue_id=1478 or queue_id=1477 or queue_id=1475 or queue_id=1434 or queue_id=1433 or queue_id=1013 ; SELECT * FROM cloud.sync_queue where id=1433 or id=1434 or id=1475 or id=1477 or id=1478 or id=981; On 1/6/15 1:27 AM, Prashant Kumar Mishra wrote: Restart MS should work . If you don¹t want to restart clean it manually from db , it seems because of one unfinished job , other jobs is also pending . On 1/6/15, 4:09 AM, "Francois Gaudreault" wrote: These are 60min and 1440min for standard jobs and 600sec for vm related jobs right? I have tasks that are from dec 27 still in there and these are vm jobs. FG On Jan 5, 2015 5:25 PM, "Motty Cruz" wrote: Francois, you can change timeout default settings in "global settings", if you have not done so. Thanks, Motty On 01/05/2015 02:09 PM, Francois Gaudreault wrote: Yes. And also tons of job in the async_job table with status 0. FG On 2015-01-05 4:54 PM, ilya musayev wrote: Do you see something along the lines of below? please provide the complete error message. (1434,VmWorkJobQueue, 2471) is reaching concurrency limit 1 (1475,VmWorkJobQueue, 2483) is reaching concurrency limit 1 (1477,VmWorkJobQueue, 2484) is reaching concurrency limit 1 (1478,VmWorkJobQueue, 596) is reaching concurrency limit 1 (981,VmWorkJobQueue, 2371) is reaching concurrency limit 1 On 1/5/15, 1:40 PM, Francois Gaudreault wrote: Hi folks, What would be the best way to cancel jobs that are "stuck"? I was expecting them to timeout, but for some reasons they are not. Thanks. -- Francois Gaudreault Gestionnaire de Produit | Product Manager - Cloud Platform & Services t:514-629-6775 CloudOps Votre partenaire infonuagique | Cloud Solutions Experts 420 rue Guy | Montreal | Quebec | H3J 1S6 w: cloudops.com | tw: @CloudOps_
Re: Cancel hung jobs
Here are my notes on cleaning up stuck VmWorkJobQueue, i use MySQL workbench to make changes interactively... Messages in log file: (1013,VmWorkJobQueue, 2375) is reaching concurrency limit 1 (1013,VmWorkJobQueue, 2375) is reaching concurrency limit 1 (1433,VmWorkJobQueue, 2470) is reaching concurrency limit 1 (1434,VmWorkJobQueue, 2471) is reaching concurrency limit 1 (1475,VmWorkJobQueue, 2483) is reaching concurrency limit 1 (1477,VmWorkJobQueue, 2484) is reaching concurrency limit 1 (1478,VmWorkJobQueue, 596) is reaching concurrency limit 1 (981,VmWorkJobQueue, 2371) is reaching concurrency limit 1 -- DELETE ALL FROM BELOW - not exact mysql query, i just did select first and then deleted rows via mysql workbench. SELECT * FROM cloud.sync_queue_item where queue_id=981 or queue_id=1478 or queue_id=1477 or queue_id=1475 or queue_id=1434 or queue_id=1433 or queue_id=1013 ; SELECT * FROM cloud.sync_queue where id=1433 or id=1434 or id=1475 or id=1477 or id=1478 or id=981; On 1/6/15 1:27 AM, Prashant Kumar Mishra wrote: Restart MS should work . If you don¹t want to restart clean it manually from db , it seems because of one unfinished job , other jobs is also pending . On 1/6/15, 4:09 AM, "Francois Gaudreault" wrote: These are 60min and 1440min for standard jobs and 600sec for vm related jobs right? I have tasks that are from dec 27 still in there and these are vm jobs. FG On Jan 5, 2015 5:25 PM, "Motty Cruz" wrote: Francois, you can change timeout default settings in "global settings", if you have not done so. Thanks, Motty On 01/05/2015 02:09 PM, Francois Gaudreault wrote: Yes. And also tons of job in the async_job table with status 0. FG On 2015-01-05 4:54 PM, ilya musayev wrote: Do you see something along the lines of below? please provide the complete error message. (1434,VmWorkJobQueue, 2471) is reaching concurrency limit 1 (1475,VmWorkJobQueue, 2483) is reaching concurrency limit 1 (1477,VmWorkJobQueue, 2484) is reaching concurrency limit 1 (1478,VmWorkJobQueue, 596) is reaching concurrency limit 1 (981,VmWorkJobQueue, 2371) is reaching concurrency limit 1 On 1/5/15, 1:40 PM, Francois Gaudreault wrote: Hi folks, What would be the best way to cancel jobs that are "stuck"? I was expecting them to timeout, but for some reasons they are not. Thanks.
Re: Cancel hung jobs
Restart MS should work . If you don¹t want to restart clean it manually from db , it seems because of one unfinished job , other jobs is also pending . On 1/6/15, 4:09 AM, "Francois Gaudreault" wrote: >These are 60min and 1440min for standard jobs and 600sec for vm related >jobs right? I have tasks that are from dec 27 still in there and these >are vm jobs. > >FG >On Jan 5, 2015 5:25 PM, "Motty Cruz" wrote: > >> Francois, >> you can change timeout default settings in "global settings", if you >>have >> not done so. >> >> Thanks, >> Motty >> On 01/05/2015 02:09 PM, Francois Gaudreault wrote: >> >>> Yes. And also tons of job in the async_job table with status 0. >>> >>> FG >>> >>> On 2015-01-05 4:54 PM, ilya musayev wrote: >>> Do you see something along the lines of below? please provide the complete error message. (1434,VmWorkJobQueue, 2471) is reaching concurrency limit 1 (1475,VmWorkJobQueue, 2483) is reaching concurrency limit 1 (1477,VmWorkJobQueue, 2484) is reaching concurrency limit 1 (1478,VmWorkJobQueue, 596) is reaching concurrency limit 1 (981,VmWorkJobQueue, 2371) is reaching concurrency limit 1 On 1/5/15, 1:40 PM, Francois Gaudreault wrote: > Hi folks, > > What would be the best way to cancel jobs that are "stuck"? I was > expecting them to timeout, but for some reasons they are not. > > Thanks. > > >>> >>> >>
Re: Cancel hung jobs
These are 60min and 1440min for standard jobs and 600sec for vm related jobs right? I have tasks that are from dec 27 still in there and these are vm jobs. FG On Jan 5, 2015 5:25 PM, "Motty Cruz" wrote: > Francois, > you can change timeout default settings in "global settings", if you have > not done so. > > Thanks, > Motty > On 01/05/2015 02:09 PM, Francois Gaudreault wrote: > >> Yes. And also tons of job in the async_job table with status 0. >> >> FG >> >> On 2015-01-05 4:54 PM, ilya musayev wrote: >> >>> Do you see something along the lines of below? please provide the >>> complete error message. >>> >>> (1434,VmWorkJobQueue, 2471) is reaching concurrency limit 1 >>> (1475,VmWorkJobQueue, 2483) is reaching concurrency limit 1 >>> (1477,VmWorkJobQueue, 2484) is reaching concurrency limit 1 >>> (1478,VmWorkJobQueue, 596) is reaching concurrency limit 1 >>> (981,VmWorkJobQueue, 2371) is reaching concurrency limit 1 >>> >>> >>> On 1/5/15, 1:40 PM, Francois Gaudreault wrote: >>> Hi folks, What would be the best way to cancel jobs that are "stuck"? I was expecting them to timeout, but for some reasons they are not. Thanks. >>> >>> >>> >> >> >
Re: Cancel hung jobs
Francois, you can change timeout default settings in "global settings", if you have not done so. Thanks, Motty On 01/05/2015 02:09 PM, Francois Gaudreault wrote: Yes. And also tons of job in the async_job table with status 0. FG On 2015-01-05 4:54 PM, ilya musayev wrote: Do you see something along the lines of below? please provide the complete error message. (1434,VmWorkJobQueue, 2471) is reaching concurrency limit 1 (1475,VmWorkJobQueue, 2483) is reaching concurrency limit 1 (1477,VmWorkJobQueue, 2484) is reaching concurrency limit 1 (1478,VmWorkJobQueue, 596) is reaching concurrency limit 1 (981,VmWorkJobQueue, 2371) is reaching concurrency limit 1 On 1/5/15, 1:40 PM, Francois Gaudreault wrote: Hi folks, What would be the best way to cancel jobs that are "stuck"? I was expecting them to timeout, but for some reasons they are not. Thanks.
Re: Cancel hung jobs
Yes. And also tons of job in the async_job table with status 0. FG On 2015-01-05 4:54 PM, ilya musayev wrote: Do you see something along the lines of below? please provide the complete error message. (1434,VmWorkJobQueue, 2471) is reaching concurrency limit 1 (1475,VmWorkJobQueue, 2483) is reaching concurrency limit 1 (1477,VmWorkJobQueue, 2484) is reaching concurrency limit 1 (1478,VmWorkJobQueue, 596) is reaching concurrency limit 1 (981,VmWorkJobQueue, 2371) is reaching concurrency limit 1 On 1/5/15, 1:40 PM, Francois Gaudreault wrote: Hi folks, What would be the best way to cancel jobs that are "stuck"? I was expecting them to timeout, but for some reasons they are not. Thanks. -- Francois Gaudreault Gestionnaire de Produit | Product Manager - Cloud Platform & Services t:514-629-6775 CloudOps Votre partenaire infonuagique | Cloud Solutions Experts 420 rue Guy | Montreal | Quebec | H3J 1S6 w: cloudops.com | tw: @CloudOps_
Re: Cancel hung jobs
Do you see something along the lines of below? please provide the complete error message. (1434,VmWorkJobQueue, 2471) is reaching concurrency limit 1 (1475,VmWorkJobQueue, 2483) is reaching concurrency limit 1 (1477,VmWorkJobQueue, 2484) is reaching concurrency limit 1 (1478,VmWorkJobQueue, 596) is reaching concurrency limit 1 (981,VmWorkJobQueue, 2371) is reaching concurrency limit 1 On 1/5/15, 1:40 PM, Francois Gaudreault wrote: Hi folks, What would be the best way to cancel jobs that are "stuck"? I was expecting them to timeout, but for some reasons they are not. Thanks.
Cancel hung jobs
Hi folks, What would be the best way to cancel jobs that are "stuck"? I was expecting them to timeout, but for some reasons they are not. Thanks. -- Francois Gaudreault Gestionnaire de Produit | Product Manager - Cloud Platform & Services t:514-629-6775 CloudOps Votre partenaire infonuagique | Cloud Solutions Experts 420 rue Guy | Montreal | Quebec | H3J 1S6 w: cloudops.com | tw: @CloudOps_
