Re: Cloudstack Usage --- not owner
> > > > > > > > right ? > > > > > > > > > > > > > > > > > > > > > > i deleted the null record and restart usage servoce > and > > > see > > > > > > > > following > > > > > > > > > > in > > > > > > > > > > > usage log : > > > > > > > > > > > > > > > > > > > > > > 2021-06-19 18:37:52,912 INFO > > > [cloud.usage.UsageManagerImpl] > > > > > > > > > (main:null) > > > > > > > > > > > (logid:) Implementation Version is 4.15.0.0 > > > > > > > > > > > > > > > > > > > > > > 2021-06-19 18:37:55,942 DEBUG > > > [cloud.usage.UsageManagerImpl] > > > > > > > > > (main:null) > > > > > > > > > > > (logid:) Usage stats aggregation time zone: GMT > > > > > > > > > > > > > > > > > > > > > > 2021-06-19 18:37:55,952 DEBUG > > > [cloud.usage.UsageManagerImpl] > > > > > > > > > (main:null) > > > > > > > > > > > (logid:) Execution Time: Sat Jun 19 00:15:00 UTC 2021 > > > > > > > > > > > > > > > > > > > > > > 2021-06-19 18:37:55,953 DEBUG > > > [cloud.usage.UsageManagerImpl] > > > > > > > > > (main:null) > > > > > > > > > > > (logid:) Current Time: Sat Jun 19 18:37:55 UTC 2021 > > > > > > > > > > > > > > > > > > > > > > It seem execution time before current time ? Is that > > > normal > > > > ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Jun 20, 2021 at 2:39 AM Hean Seng < > > > > heans...@gmail.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, I shall stop the Cloudstack-Usage services and > > delete > > > > all > > > > > > > > record > > > > > > > > > > > where > > > > > > > > > > > > success is NULL , and start back the usage service ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Jun 19, 2021 at 11:00 PM Daan Hoogland < > > > > > > > > > > daan.hoogl...@gmail.com> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > >> Heanm, > > > > > > > > > > > >> > > > > > > > > > > > >> On Sat, Jun 19, 2021 at 5:27 AM Hean Seng < > > > > > heans...@gmail.com > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > >> > > > > > > > > > > > >> > i have no unprocess job , but still have this > issue: > > > > > > > > > > > >> > > > > > > > > > > > > >> > 2021-06-19 10:09:57,657 DEBUG > > > > > [cloud.usage.UsageManagerImpl] > > > > > > > > > > > >> > (Usage-HB-1:null) (logid:) Scheduling Usage job... > > > > > > > > > > > >> > > > > > > > > > > > > >> > 2021-06-19 10:09:57,657 INFO > > > > > [cloud.usage.UsageManagerImpl] > > > > > > > > > > > >> > (Usage-Job-1:null) (logid:) starting usage job... > > > > > > > > > > > >> > > > > > > > > > > > > >> > 2021-06-19 10:09:57,661 DEBUG > > > > > [cloud.usage.UsageManagerImpl] > > > > > > > > > > > >> > (Usage-Job-1:null) (logid:) Not owner of usage > job, > > > > > > > skipping... > > > > > > > > > > > >> > > > > > > > > > > > > >> this line ^^ means there is a unprocessed job owned > by > > > > > another > > > > > > > > > > proccess > > > > > > > > > > > >> ... > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > 2021-06-19 10:09:57,661 INFO > > > > > [cloud.usage.UsageManagerImpl] > > > > > > > > > > > >> > (Usage-Job-1:null) (logid:) usage job complete > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > >> > mysql> select id, pid, start_date ,end_date , > > > success, > > > > > > > > heartbeat > > > > > > > > > > from > > > > > > > > > > > >> > usage_job where success=0 order by start_date desc > > > limit > > > > > > 100; > > > > > > > > > > > >> > > > > > > > > > > > > >> you should search for `where success is null` > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > Empty set (0.00 sec) > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > >> > On Fri, Jun 18, 2021 at 11:01 PM Hean Seng < > > > > > > > heans...@gmail.com> > > > > > > > > > > > wrote: > > > > > > > > > > > >> > > > > > > > > > > > > >> > > yes, I restarted the mgmt server, wait till > can > > > > login > > > > > > web > > > > > > > > > > > interface > > > > > > > > > > > >> . > > > > > > > > > > > >> > > then only restart usage . > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > I did follwoig also : > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > * stop usage server > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > * remove the record from usage_job for the > > > unprocessed > > > > > job > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > * start usage server again > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > Still same thing. Not sure where the Cloudstack > > > keep > > > > > the > > > > > > > > Usage > > > > > > > > > > > server > > > > > > > > > > > >> > PID > > > > > > > > > > > >> > > id , and keep getting wrong id. > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > On Fri, Jun 18, 2021 at 4:31 PM Boris Stoyanov < > > > > > > > > > > > >> > > boris.stoya...@shapeblue.com> wrote: > > > > > > > > > > > >> > > > > > > > > > > > > > >> > >> Hi Hean Seng, > > > > > > > > > > > >> > >> > > > > > > > > > > > >> > >> Have you waited the management service to come > up > > > and > > > > > > then > > > > > > > > > > > restarted > > > > > > > > > > > >> > >> usage service? It might be some race condition? > > > > > > > > > > > >> > >> > > > > > > > > > > > >> > >> Bobby. > > > > > > > > > > > >> > >> > > > > > > > > > > > >> > >> From: Hean Seng > > > > > > > > > > > >> > >> Date: Friday, 18 June 2021, 7:31 > > > > > > > > > > > >> > >> To: users@cloudstack.apache.org < > > > > > > > users@cloudstack.apache.org > > > > > > > > > > > > > > > > > > > > >> > >> Subject: Cloudstack Usage --- not owner > > > > > > > > > > > >> > >> Hi > > > > > > > > > > > >> > >> > > > > > > > > > > > >> > >> I have mysql hang, and restarted mysql, > restarted > > > > > > > cloudstack > > > > > > > > > > mgmt, > > > > > > > > > > > >> and > > > > > > > > > > > >> > >> usage. > > > > > > > > > > > >> > >> > > > > > > > > > > > >> > >> After that facing issue of following: > > > > > > > > > > > >> > >> > > > > > > > > > > > >> > >> duration is 120 minutes) > > > > > > > > > > > >> > >> > > > > > > > > > > > >> > >> 2021-06-18 04:25:49,656 DEBUG > > > > > > > [cloud.usage.UsageManagerImpl] > > > > > > > > > > > >> > >> (Usage-HB-1:null) (logid:) Scheduling Usage > > job... > > > > > > > > > > > >> > >> > > > > > > > > > > > >> > >> 2021-06-18 04:25:49,657 INFO > > > > > > > [cloud.usage.UsageManagerImpl] > > > > > > > > > > > >> > >> (Usage-Job-1:null) (logid:) starting usage > job... > > > > > > > > > > > >> > >> > > > > > > > > > > > >> > >> 2021-06-18 04:25:49,669 DEBUG > > > > > > > [cloud.usage.UsageManagerImpl] > > > > > > > > > > > >> > >> (Usage-Job-1:null) (logid:) Not owner of usage > > job, > > > > > > > > skipping... > > > > > > > > > > > >> > >> > > > > > > > > > > > >> > >> 2021-06-18 04:25:49,669 INFO > > > > > > > [cloud.usage.UsageManagerImpl] > > > > > > > > > > > >> > >> (Usage-Job-1:null) (logid:) usage job complete > > > > > > > > > > > >> > >> > > > > > > > > > > > >> > >> > > > > > > > > > > > >> > >> > > > > > > > > > > > >> > >> Seems usage pid is not updated to db . > > > > > > > > > > > >> > >> > > > > > > > > > > > >> > >> Andybody know how to fix this ? > > > > > > > > > > > >> > >> > > > > > > > > > > > >> > >> -- > > > > > > > > > > > >> > >> Regards, > > > > > > > > > > > >> > >> Hean Seng > > > > > > > > > > > >> > >> > > > > > > > > > > > >> > >> > > > > > > > > > > > >> > >> > > > > > > > > > > > >> > >> > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > -- > > > > > > > > > > > >> > > Regards, > > > > > > > > > > > >> > > Hean Seng > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > >> > -- > > > > > > > > > > > >> > Regards, > > > > > > > > > > > >> > Hean Seng > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> -- > > > > > > > > > > > >> Daan > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > Regards, > > > > > > > > > > > > Hean Seng > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > Regards, > > > > > > > > > > > Hean Seng > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Daan > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Regards, > > > > > > > > > Hean Seng > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Daan > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Regards, > > > > > > > Hean Seng > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Daan > > > > > > > > > > > > > > > > > > > > > -- > > > > > Regards, > > > > > Hean Seng > > > > > > > > > > > > > > > > > -- > > > > Daan > > > > > > > > > > > > > -- > > > Regards, > > > Hean Seng > > > > > > > > > -- > > > > Andrija Panić > > > > > -- > Regards, > Hean Seng > -- Daan
Re: Cloudstack Usage --- not owner
The it will auto generate back the record with > correct > > > ID . > > > > > > > > > > > > > > > > > > > > > > > > However, I am facing the issue is the Usage Type 4, and 5 > > > > (Network > > > > > > sent > > > > > > > > and received) are not populated . Other Type like Type1 is > > > always > > > > > > > working. > > > > > > > > > > > > > > > > Fyi, I am using share network, and set the parameter > > > > > > > > vm.network.stats.interval=300 > > > > > > > > so that the VM statistic is updated . > > > > > > > > > > > > > > > > This issue not happen on all VM, but part of it. ( about > 50%) . > > > I > > > > > > > > tried look at the DB, and seem cannot find the clue what > > actually > > > > > diff > > > > > > , > > > > > > > in > > > > > > > > term of those updated and those is not. > > > > > > > > > > > > > > > > The record is appearing in usage_network_offering and seems > > good. > > > > Do > > > > > > you > > > > > > > > have any idea of this ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Jun 21, 2021 at 3:19 PM Daan Hoogland < > > > > > daan.hoogl...@gmail.com > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Sorry Hean, wasn't on line most of the weekend, Is it > running > > > > now? > > > > > > > > > as you wrote this two days ago, it should already have run > > > twice > > > > > > > > (assuming > > > > > > > > > you didn't change a lot) > > > > > > > > > > > > > > > > > > On Sat, Jun 19, 2021 at 8:41 PM Hean Seng < > > heans...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > > > > > > And it will insert another success is NULL to usage_job > > > table , > > > > > > > right ? > > > > > > > > > > > > > > > > > > > > i deleted the null record and restart usage servoce and > > see > > > > > > > following > > > > > > > > > in > > > > > > > > > > usage log : > > > > > > > > > > > > > > > > > > > > 2021-06-19 18:37:52,912 INFO > > [cloud.usage.UsageManagerImpl] > > > > > > > > (main:null) > > > > > > > > > > (logid:) Implementation Version is 4.15.0.0 > > > > > > > > > > > > > > > > > > > > 2021-06-19 18:37:55,942 DEBUG > > [cloud.usage.UsageManagerImpl] > > > > > > > > (main:null) > > > > > > > > > > (logid:) Usage stats aggregation time zone: GMT > > > > > > > > > > > > > > > > > > > > 2021-06-19 18:37:55,952 DEBUG > > [cloud.usage.UsageManagerImpl] > > > > > > > > (main:null) > > > > > > > > > > (logid:) Execution Time: Sat Jun 19 00:15:00 UTC 2021 > > > > > > > > > > > > > > > > > > > > 2021-06-19 18:37:55,953 DEBUG > > [cloud.usage.UsageManagerImpl] > > > > > > > > (main:null) > > > > > > > > > > (logid:) Current Time: Sat Jun 19 18:37:55 UTC 2021 > > > > > > > > > > > > > > > > > > > > It seem execution time before current time ? Is that > > normal > > > ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > &
Re: Cloudstack Usage --- not owner
> > > > > > > > > > > > > On Mon, Jun 21, 2021 at 3:19 PM Daan Hoogland < > > > > daan.hoogl...@gmail.com > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Sorry Hean, wasn't on line most of the weekend, Is it running > > > now? > > > > > > > > as you wrote this two days ago, it should already have run > > twice > > > > > > > (assuming > > > > > > > > you didn't change a lot) > > > > > > > > > > > > > > > > On Sat, Jun 19, 2021 at 8:41 PM Hean Seng < > heans...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > > > > > And it will insert another success is NULL to usage_job > > table , > > > > > > right ? > > > > > > > > > > > > > > > > > > i deleted the null record and restart usage servoce and > see > > > > > > following > > > > > > > > in > > > > > > > > > usage log : > > > > > > > > > > > > > > > > > > 2021-06-19 18:37:52,912 INFO > [cloud.usage.UsageManagerImpl] > > > > > > > (main:null) > > > > > > > > > (logid:) Implementation Version is 4.15.0.0 > > > > > > > > > > > > > > > > > > 2021-06-19 18:37:55,942 DEBUG > [cloud.usage.UsageManagerImpl] > > > > > > > (main:null) > > > > > > > > > (logid:) Usage stats aggregation time zone: GMT > > > > > > > > > > > > > > > > > > 2021-06-19 18:37:55,952 DEBUG > [cloud.usage.UsageManagerImpl] > > > > > > > (main:null) > > > > > > > > > (logid:) Execution Time: Sat Jun 19 00:15:00 UTC 2021 > > > > > > > > > > > > > > > > > > 2021-06-19 18:37:55,953 DEBUG > [cloud.usage.UsageManagerImpl] > > > > > > > (main:null) > > > > > > > > > (logid:) Current Time: Sat Jun 19 18:37:55 UTC 2021 > > > > > > > > > > > > > > > > > > It seem execution time before current time ? Is that > normal > > ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Jun 20, 2021 at 2:39 AM Hean Seng < > > heans...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, I shall stop the Cloudstack-Usage services and delete > > all > > > > > > record > > > > > > > > > where > > > > > > > > > > success is NULL , and start back the usage service ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Jun 19, 2021 at 11:00 PM Daan Hoogland < > > > > > > > > daan.hoogl...@gmail.com> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > >> Heanm, > > > > > > > > > >> > > > > > > > > > >> On Sat, Jun 19, 2021 at 5:27 AM Hean Seng < > > > heans...@gmail.com > > > > > > > > > > > > wrote: > > > > > > > > > >> > > > > > > > > > >> > i have no unprocess job , but still have this issue: > > > > > > > > > >> > > > > > > > > > > >> > 2021-06-19 10:09:57,657 DEBUG > > > [cloud.usage.UsageManagerImpl] > > > > > > > > > >> > (Usage-HB-1:null) (logid:) Scheduling Usage job... > > > > > > > > > >> > > > > > > > > > > >> > 2021-06-19 10:09:57,657 INFO > > > [cloud.usage.UsageManagerImpl] > > > > > > > > > >> > (Usage-Job-1:null) (logid:) starting usage job... > > > > >
Re: Cloudstack Usage --- not owner
0.0 > > > > > > > > > > > > > > > > 2021-06-19 18:37:55,942 DEBUG [cloud.usage.UsageManagerImpl] > > > > > > (main:null) > > > > > > > > (logid:) Usage stats aggregation time zone: GMT > > > > > > > > > > > > > > > > 2021-06-19 18:37:55,952 DEBUG [cloud.usage.UsageManagerImpl] > > > > > > (main:null) > > > > > > > > (logid:) Execution Time: Sat Jun 19 00:15:00 UTC 2021 > > > > > > > > > > > > > > > > 2021-06-19 18:37:55,953 DEBUG [cloud.usage.UsageManagerImpl] > > > > > > (main:null) > > > > > > > > (logid:) Current Time: Sat Jun 19 18:37:55 UTC 2021 > > > > > > > > > > > > > > > > It seem execution time before current time ? Is that normal > ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Jun 20, 2021 at 2:39 AM Hean Seng < > heans...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > So, I shall stop the Cloudstack-Usage services and delete > all > > > > > record > > > > > > > > where > > > > > > > > > success is NULL , and start back the usage service ? > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Jun 19, 2021 at 11:00 PM Daan Hoogland < > > > > > > > daan.hoogl...@gmail.com> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > >> Heanm, > > > > > > > > >> > > > > > > > > >> On Sat, Jun 19, 2021 at 5:27 AM Hean Seng < > > heans...@gmail.com > > > > > > > > > > wrote: > > > > > > > > >> > > > > > > > > >> > i have no unprocess job , but still have this issue: > > > > > > > > >> > > > > > > > > > >> > 2021-06-19 10:09:57,657 DEBUG > > [cloud.usage.UsageManagerImpl] > > > > > > > > >> > (Usage-HB-1:null) (logid:) Scheduling Usage job... > > > > > > > > >> > > > > > > > > > >> > 2021-06-19 10:09:57,657 INFO > > [cloud.usage.UsageManagerImpl] > > > > > > > > >> > (Usage-Job-1:null) (logid:) starting usage job... > > > > > > > > >> > > > > > > > > > >> > 2021-06-19 10:09:57,661 DEBUG > > [cloud.usage.UsageManagerImpl] > > > > > > > > >> > (Usage-Job-1:null) (logid:) Not owner of usage job, > > > > skipping... > > > > > > > > >> > > > > > > > > > >> this line ^^ means there is a unprocessed job owned by > > another > > > > > > > proccess > > > > > > > > >> ... > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > 2021-06-19 10:09:57,661 INFO > > [cloud.usage.UsageManagerImpl] > > > > > > > > >> > (Usage-Job-1:null) (logid:) usage job complete > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > mysql> select id, pid, start_date ,end_date , success, > > > > > heartbeat > > > > > > > from > > > > > > > > >> > usage_job where success=0 order by start_date desc limit > > > 100; > > > > > > > > >> > > > > > > > > > >> you should search for `where success is null` > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > Empty set (0.00 sec) > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > On Fri, Jun 18, 2021 at 11:01 PM Hean Seng < > > > > heans...@gmail.com> > > > > > > > > wrote: > > > > > > > > >> > > > > > > > > > >> > > yes, I restarted the mgmt server, wait till can > login > > > web > > > > > > > > interface > > > > > > > > >> . > > > > > > > > >> > > then only restart usage . > > > > > > > > >> > > > > > > > > > > >> > > I did follwoig also : > > > > > > > > >> > > > > > > > > > > >> > > * stop usage server > > > > > > > > >> > > > > > > > > > > >> > > * remove the record from usage_job for the unprocessed > > job > > > > > > > > >> > > > > > > > > > > >> > > * start usage server again > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > Still same thing. Not sure where the Cloudstack keep > > the > > > > > Usage > > > > > > > > server > > > > > > > > >> > PID > > > > > > > > >> > > id , and keep getting wrong id. > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > On Fri, Jun 18, 2021 at 4:31 PM Boris Stoyanov < > > > > > > > > >> > > boris.stoya...@shapeblue.com> wrote: > > > > > > > > >> > > > > > > > > > > >> > >> Hi Hean Seng, > > > > > > > > >> > >> > > > > > > > > >> > >> Have you waited the management service to come up and > > > then > > > > > > > > restarted > > > > > > > > >> > >> usage service? It might be some race condition? > > > > > > > > >> > >> > > > > > > > > >> > >> Bobby. > > > > > > > > >> > >> > > > > > > > > >> > >> From: Hean Seng > > > > > > > > >> > >> Date: Friday, 18 June 2021, 7:31 > > > > > > > > >> > >> To: users@cloudstack.apache.org < > > > > users@cloudstack.apache.org > > > > > > > > > > > > > > >> > >> Subject: Cloudstack Usage --- not owner > > > > > > > > >> > >> Hi > > > > > > > > >> > >> > > > > > > > > >> > >> I have mysql hang, and restarted mysql, restarted > > > > cloudstack > > > > > > > mgmt, > > > > > > > > >> and > > > > > > > > >> > >> usage. > > > > > > > > >> > >> > > > > > > > > >> > >> After that facing issue of following: > > > > > > > > >> > >> > > > > > > > > >> > >> duration is 120 minutes) > > > > > > > > >> > >> > > > > > > > > >> > >> 2021-06-18 04:25:49,656 DEBUG > > > > [cloud.usage.UsageManagerImpl] > > > > > > > > >> > >> (Usage-HB-1:null) (logid:) Scheduling Usage job... > > > > > > > > >> > >> > > > > > > > > >> > >> 2021-06-18 04:25:49,657 INFO > > > > [cloud.usage.UsageManagerImpl] > > > > > > > > >> > >> (Usage-Job-1:null) (logid:) starting usage job... > > > > > > > > >> > >> > > > > > > > > >> > >> 2021-06-18 04:25:49,669 DEBUG > > > > [cloud.usage.UsageManagerImpl] > > > > > > > > >> > >> (Usage-Job-1:null) (logid:) Not owner of usage job, > > > > > skipping... > > > > > > > > >> > >> > > > > > > > > >> > >> 2021-06-18 04:25:49,669 INFO > > > > [cloud.usage.UsageManagerImpl] > > > > > > > > >> > >> (Usage-Job-1:null) (logid:) usage job complete > > > > > > > > >> > >> > > > > > > > > >> > >> > > > > > > > > >> > >> > > > > > > > > >> > >> Seems usage pid is not updated to db . > > > > > > > > >> > >> > > > > > > > > >> > >> Andybody know how to fix this ? > > > > > > > > >> > >> > > > > > > > > >> > >> -- > > > > > > > > >> > >> Regards, > > > > > > > > >> > >> Hean Seng > > > > > > > > >> > >> > > > > > > > > >> > >> > > > > > > > > >> > >> > > > > > > > > >> > >> > > > > > > > > >> > > > > > > > > > > >> > > -- > > > > > > > > >> > > Regards, > > > > > > > > >> > > Hean Seng > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > -- > > > > > > > > >> > Regards, > > > > > > > > >> > Hean Seng > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> -- > > > > > > > > >> Daan > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Regards, > > > > > > > > > Hean Seng > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Regards, > > > > > > > > Hean Seng > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Daan > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Regards, > > > > > > Hean Seng > > > > > > > > > > > > > > > > > > > > > -- > > > > > Daan > > > > > > > > > > > > > > > > > -- > > > > Regards, > > > > Hean Seng > > > > > > > > > > > > > -- > > > Daan > > > > > > > > > -- > > Regards, > > Hean Seng > > > > > -- > Daan > -- Regards, Hean Seng
Re: Cloudstack Usage --- not owner
es and delete all > > > > record > > > > > > > where > > > > > > > > success is NULL , and start back the usage service ? > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Jun 19, 2021 at 11:00 PM Daan Hoogland < > > > > > > daan.hoogl...@gmail.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > >> Heanm, > > > > > > > >> > > > > > > > >> On Sat, Jun 19, 2021 at 5:27 AM Hean Seng < > heans...@gmail.com > > > > > > > > wrote: > > > > > > > >> > > > > > > > >> > i have no unprocess job , but still have this issue: > > > > > > > >> > > > > > > > > >> > 2021-06-19 10:09:57,657 DEBUG > [cloud.usage.UsageManagerImpl] > > > > > > > >> > (Usage-HB-1:null) (logid:) Scheduling Usage job... > > > > > > > >> > > > > > > > > >> > 2021-06-19 10:09:57,657 INFO > [cloud.usage.UsageManagerImpl] > > > > > > > >> > (Usage-Job-1:null) (logid:) starting usage job... > > > > > > > >> > > > > > > > > >> > 2021-06-19 10:09:57,661 DEBUG > [cloud.usage.UsageManagerImpl] > > > > > > > >> > (Usage-Job-1:null) (logid:) Not owner of usage job, > > > skipping... > > > > > > > >> > > > > > > > > >> this line ^^ means there is a unprocessed job owned by > another > > > > > > proccess > > > > > > > >> ... > > > > > > > >> > > > > > > > >> > > > > > > > >> > 2021-06-19 10:09:57,661 INFO > [cloud.usage.UsageManagerImpl] > > > > > > > >> > (Usage-Job-1:null) (logid:) usage job complete > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > mysql> select id, pid, start_date ,end_date , success, > > > > heartbeat > > > > > > from > > > > > > > >> > usage_job where success=0 order by start_date desc limit > > 100; > > > > > > > >> > > > > > > > > >> you should search for `where success is null` > > > > > > > >> > > > > > > > >> > > > > > > > >> > Empty set (0.00 sec) > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > On Fri, Jun 18, 2021 at 11:01 PM Hean Seng < > > > heans...@gmail.com> > > > > > > > wrote: > > > > > > > >> > > > > > > > > >> > > yes, I restarted the mgmt server, wait till can login > > web > > > > > > > interface > > > > > > > >> . > > > > > > > >> > > then only restart usage . > > > > > > > >> > > > > > > > > > >> > > I did follwoig also : > > > > > > > >> > > > > > > > > > >> > > * stop usage server > > > > > > > >> > > > > > > > > > >> > > * remove the record from usage_job for the unprocessed > job > > > > > > > >> > > > > > > > > > >> > > * start usage server again > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > Still same thing. Not sure where the Cloudstack keep > the > > > > Usage > > > > > > > server > > > > > > > >> > PID > > > > > > > >> > > id , and keep getting wrong id. > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > On Fri, Jun 18, 2021 at 4:31 PM Boris Stoyanov < > > > > > > > >> > > boris.stoya...@shapeblue.com> wrote: > > > > > > > >> > > > > > > > > > >> > >> Hi Hean Seng, > > > > > > > >> > >> > > > > > > > >> > >> Have you waited the management service to come up and > > then > > > > > > > restarted > > > > > > > >> > >> usage service? It might be some race condition? > > > > > > > >> > >> > > > > > > > >> > >> Bobby. > > > > > > > >> > >> > > > > > > > >> > >> From: Hean Seng > > > > > > > >> > >> Date: Friday, 18 June 2021, 7:31 > > > > > > > >> > >> To: users@cloudstack.apache.org < > > > users@cloudstack.apache.org > > > > > > > > > > > > >> > >> Subject: Cloudstack Usage --- not owner > > > > > > > >> > >> Hi > > > > > > > >> > >> > > > > > > > >> > >> I have mysql hang, and restarted mysql, restarted > > > cloudstack > > > > > > mgmt, > > > > > > > >> and > > > > > > > >> > >> usage. > > > > > > > >> > >> > > > > > > > >> > >> After that facing issue of following: > > > > > > > >> > >> > > > > > > > >> > >> duration is 120 minutes) > > > > > > > >> > >> > > > > > > > >> > >> 2021-06-18 04:25:49,656 DEBUG > > > [cloud.usage.UsageManagerImpl] > > > > > > > >> > >> (Usage-HB-1:null) (logid:) Scheduling Usage job... > > > > > > > >> > >> > > > > > > > >> > >> 2021-06-18 04:25:49,657 INFO > > > [cloud.usage.UsageManagerImpl] > > > > > > > >> > >> (Usage-Job-1:null) (logid:) starting usage job... > > > > > > > >> > >> > > > > > > > >> > >> 2021-06-18 04:25:49,669 DEBUG > > > [cloud.usage.UsageManagerImpl] > > > > > > > >> > >> (Usage-Job-1:null) (logid:) Not owner of usage job, > > > > skipping... > > > > > > > >> > >> > > > > > > > >> > >> 2021-06-18 04:25:49,669 INFO > > > [cloud.usage.UsageManagerImpl] > > > > > > > >> > >> (Usage-Job-1:null) (logid:) usage job complete > > > > > > > >> > >> > > > > > > > >> > >> > > > > > > > >> > >> > > > > > > > >> > >> Seems usage pid is not updated to db . > > > > > > > >> > >> > > > > > > > >> > >> Andybody know how to fix this ? > > > > > > > >> > >> > > > > > > > >> > >> -- > > > > > > > >> > >> Regards, > > > > > > > >> > >> Hean Seng > > > > > > > >> > >> > > > > > > > >> > >> > > > > > > > >> > >> > > > > > > > >> > >> > > > > > > > >> > > > > > > > > > >> > > -- > > > > > > > >> > > Regards, > > > > > > > >> > > Hean Seng > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > -- > > > > > > > >> > Regards, > > > > > > > >> > Hean Seng > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > >> -- > > > > > > > >> Daan > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Regards, > > > > > > > > Hean Seng > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Regards, > > > > > > > Hean Seng > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Daan > > > > > > > > > > > > > > > > > > > > > -- > > > > > Regards, > > > > > Hean Seng > > > > > > > > > > > > > > > > > -- > > > > Daan > > > > > > > > > > > > > -- > > > Regards, > > > Hean Seng > > > > > > > > > -- > > Daan > > > > > -- > Regards, > Hean Seng > -- Daan
Re: Cloudstack Usage --- not owner
> > (Usage-Job-1:null) (logid:) starting usage job... > > > > > > >> > > > > > > > >> > 2021-06-19 10:09:57,661 DEBUG [cloud.usage.UsageManagerImpl] > > > > > > >> > (Usage-Job-1:null) (logid:) Not owner of usage job, > > skipping... > > > > > > >> > > > > > > > >> this line ^^ means there is a unprocessed job owned by another > > > > > proccess > > > > > > >> ... > > > > > > >> > > > > > > >> > > > > > > >> > 2021-06-19 10:09:57,661 INFO [cloud.usage.UsageManagerImpl] > > > > > > >> > (Usage-Job-1:null) (logid:) usage job complete > > > > > > >> > > > > > > > >> > > > > > > > >> > mysql> select id, pid, start_date ,end_date , success, > > > heartbeat > > > > > from > > > > > > >> > usage_job where success=0 order by start_date desc limit > 100; > > > > > > >> > > > > > > > >> you should search for `where success is null` > > > > > > >> > > > > > > >> > > > > > > >> > Empty set (0.00 sec) > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > On Fri, Jun 18, 2021 at 11:01 PM Hean Seng < > > heans...@gmail.com> > > > > > > wrote: > > > > > > >> > > > > > > > >> > > yes, I restarted the mgmt server, wait till can login > web > > > > > > interface > > > > > > >> . > > > > > > >> > > then only restart usage . > > > > > > >> > > > > > > > > >> > > I did follwoig also : > > > > > > >> > > > > > > > > >> > > * stop usage server > > > > > > >> > > > > > > > > >> > > * remove the record from usage_job for the unprocessed job > > > > > > >> > > > > > > > > >> > > * start usage server again > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > Still same thing. Not sure where the Cloudstack keep the > > > Usage > > > > > > server > > > > > > >> > PID > > > > > > >> > > id , and keep getting wrong id. > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > On Fri, Jun 18, 2021 at 4:31 PM Boris Stoyanov < > > > > > > >> > > boris.stoya...@shapeblue.com> wrote: > > > > > > >> > > > > > > > > >> > >> Hi Hean Seng, > > > > > > >> > >> > > > > > > >> > >> Have you waited the management service to come up and > then > > > > > > restarted > > > > > > >> > >> usage service? It might be some race condition? > > > > > > >> > >> > > > > > > >> > >> Bobby. > > > > > > >> > >> > > > > > > >> > >> From: Hean Seng > > > > > > >> > >> Date: Friday, 18 June 2021, 7:31 > > > > > > >> > >> To: users@cloudstack.apache.org < > > users@cloudstack.apache.org > > > > > > > > > > >> > >> Subject: Cloudstack Usage --- not owner > > > > > > >> > >> Hi > > > > > > >> > >> > > > > > > >> > >> I have mysql hang, and restarted mysql, restarted > > cloudstack > > > > > mgmt, > > > > > > >> and > > > > > > >> > >> usage. > > > > > > >> > >> > > > > > > >> > >> After that facing issue of following: > > > > > > >> > >> > > > > > > >> > >> duration is 120 minutes) > > > > > > >> > >> > > > > > > >> > >> 2021-06-18 04:25:49,656 DEBUG > > [cloud.usage.UsageManagerImpl] > > > > > > >> > >> (Usage-HB-1:null) (logid:) Scheduling Usage job... > > > > > > >> > >> > > > > > > >> > >> 2021-06-18 04:25:49,657 INFO > > [cloud.usage.UsageManagerImpl] > > > > > > >> > >> (Usage-Job-1:null) (logid:) starting usage job... > > > > > > >> > >> > > > > > > >> > >> 2021-06-18 04:25:49,669 DEBUG > > [cloud.usage.UsageManagerImpl] > > > > > > >> > >> (Usage-Job-1:null) (logid:) Not owner of usage job, > > > skipping... > > > > > > >> > >> > > > > > > >> > >> 2021-06-18 04:25:49,669 INFO > > [cloud.usage.UsageManagerImpl] > > > > > > >> > >> (Usage-Job-1:null) (logid:) usage job complete > > > > > > >> > >> > > > > > > >> > >> > > > > > > >> > >> > > > > > > >> > >> Seems usage pid is not updated to db . > > > > > > >> > >> > > > > > > >> > >> Andybody know how to fix this ? > > > > > > >> > >> > > > > > > >> > >> -- > > > > > > >> > >> Regards, > > > > > > >> > >> Hean Seng > > > > > > >> > >> > > > > > > >> > >> > > > > > > >> > >> > > > > > > >> > >> > > > > > > >> > > > > > > > > >> > > -- > > > > > > >> > > Regards, > > > > > > >> > > Hean Seng > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > >> > -- > > > > > > >> > Regards, > > > > > > >> > Hean Seng > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > >> -- > > > > > > >> Daan > > > > > > >> > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Regards, > > > > > > > Hean Seng > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Regards, > > > > > > Hean Seng > > > > > > > > > > > > > > > > > > > > > -- > > > > > Daan > > > > > > > > > > > > > > > > > -- > > > > Regards, > > > > Hean Seng > > > > > > > > > > > > > -- > > > Daan > > > > > > > > > -- > > Regards, > > Hean Seng > > > > > -- > Daan > -- Regards, Hean Seng
Re: Cloudstack Usage --- not owner
hose is not. > > > > > > The record is appearing in usage_network_offering and seems good. Do > you > > > have any idea of this ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Jun 21, 2021 at 3:19 PM Daan Hoogland > > > > wrote: > > > > > > > Sorry Hean, wasn't on line most of the weekend, Is it running now? > > > > as you wrote this two days ago, it should already have run twice > > > (assuming > > > > you didn't change a lot) > > > > > > > > On Sat, Jun 19, 2021 at 8:41 PM Hean Seng > wrote: > > > > > > > > > And it will insert another success is NULL to usage_job table , > > right ? > > > > > > > > > > i deleted the null record and restart usage servoce and see > > following > > > > in > > > > > usage log : > > > > > > > > > > 2021-06-19 18:37:52,912 INFO [cloud.usage.UsageManagerImpl] > > > (main:null) > > > > > (logid:) Implementation Version is 4.15.0.0 > > > > > > > > > > 2021-06-19 18:37:55,942 DEBUG [cloud.usage.UsageManagerImpl] > > > (main:null) > > > > > (logid:) Usage stats aggregation time zone: GMT > > > > > > > > > > 2021-06-19 18:37:55,952 DEBUG [cloud.usage.UsageManagerImpl] > > > (main:null) > > > > > (logid:) Execution Time: Sat Jun 19 00:15:00 UTC 2021 > > > > > > > > > > 2021-06-19 18:37:55,953 DEBUG [cloud.usage.UsageManagerImpl] > > > (main:null) > > > > > (logid:) Current Time: Sat Jun 19 18:37:55 UTC 2021 > > > > > > > > > > It seem execution time before current time ? Is that normal ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Jun 20, 2021 at 2:39 AM Hean Seng > > wrote: > > > > > > > > > > > > > > > > > So, I shall stop the Cloudstack-Usage services and delete all > > record > > > > > where > > > > > > success is NULL , and start back the usage service ? > > > > > > > > > > > > > > > > > > On Sat, Jun 19, 2021 at 11:00 PM Daan Hoogland < > > > > daan.hoogl...@gmail.com> > > > > > > wrote: > > > > > > > > > > > >> Heanm, > > > > > >> > > > > > >> On Sat, Jun 19, 2021 at 5:27 AM Hean Seng > > > wrote: > > > > > >> > > > > > >> > i have no unprocess job , but still have this issue: > > > > > >> > > > > > > >> > 2021-06-19 10:09:57,657 DEBUG [cloud.usage.UsageManagerImpl] > > > > > >> > (Usage-HB-1:null) (logid:) Scheduling Usage job... > > > > > >> > > > > > > >> > 2021-06-19 10:09:57,657 INFO [cloud.usage.UsageManagerImpl] > > > > > >> > (Usage-Job-1:null) (logid:) starting usage job... > > > > > >> > > > > > > >> > 2021-06-19 10:09:57,661 DEBUG [cloud.usage.UsageManagerImpl] > > > > > >> > (Usage-Job-1:null) (logid:) Not owner of usage job, > skipping... > > > > > >> > > > > > > >> this line ^^ means there is a unprocessed job owned by another > > > > proccess > > > > > >> ... > > > > > >> > > > > > >> > > > > > >> > 2021-06-19 10:09:57,661 INFO [cloud.usage.UsageManagerImpl] > > > > > >> > (Usage-Job-1:null) (logid:) usage job complete > > > > > >> > > > > > > >> > > > > > > >> > mysql> select id, pid, start_date ,end_date , success, > > heartbeat > > > > from > > > > > >> > usage_job where success=0 order by start_date desc limit 100; > > > > > >> > > > > > > >> you should search for `where success is null` > > > > > >> > > > > > >> > > > > > >> > Empty set (0.00 sec) > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >>
Re: Cloudstack Usage --- not owner
> > > > > > > > i deleted the null record and restart usage servoce and see > following > > > in > > > > usage log : > > > > > > > > 2021-06-19 18:37:52,912 INFO [cloud.usage.UsageManagerImpl] > > (main:null) > > > > (logid:) Implementation Version is 4.15.0.0 > > > > > > > > 2021-06-19 18:37:55,942 DEBUG [cloud.usage.UsageManagerImpl] > > (main:null) > > > > (logid:) Usage stats aggregation time zone: GMT > > > > > > > > 2021-06-19 18:37:55,952 DEBUG [cloud.usage.UsageManagerImpl] > > (main:null) > > > > (logid:) Execution Time: Sat Jun 19 00:15:00 UTC 2021 > > > > > > > > 2021-06-19 18:37:55,953 DEBUG [cloud.usage.UsageManagerImpl] > > (main:null) > > > > (logid:) Current Time: Sat Jun 19 18:37:55 UTC 2021 > > > > > > > > It seem execution time before current time ? Is that normal ? > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Jun 20, 2021 at 2:39 AM Hean Seng > wrote: > > > > > > > > > > > > > > So, I shall stop the Cloudstack-Usage services and delete all > record > > > > where > > > > > success is NULL , and start back the usage service ? > > > > > > > > > > > > > > > On Sat, Jun 19, 2021 at 11:00 PM Daan Hoogland < > > > daan.hoogl...@gmail.com> > > > > > wrote: > > > > > > > > > >> Heanm, > > > > >> > > > > >> On Sat, Jun 19, 2021 at 5:27 AM Hean Seng > > wrote: > > > > >> > > > > >> > i have no unprocess job , but still have this issue: > > > > >> > > > > > >> > 2021-06-19 10:09:57,657 DEBUG [cloud.usage.UsageManagerImpl] > > > > >> > (Usage-HB-1:null) (logid:) Scheduling Usage job... > > > > >> > > > > > >> > 2021-06-19 10:09:57,657 INFO [cloud.usage.UsageManagerImpl] > > > > >> > (Usage-Job-1:null) (logid:) starting usage job... > > > > >> > > > > > >> > 2021-06-19 10:09:57,661 DEBUG [cloud.usage.UsageManagerImpl] > > > > >> > (Usage-Job-1:null) (logid:) Not owner of usage job, skipping... > > > > >> > > > > > >> this line ^^ means there is a unprocessed job owned by another > > > proccess > > > > >> ... > > > > >> > > > > >> > > > > >> > 2021-06-19 10:09:57,661 INFO [cloud.usage.UsageManagerImpl] > > > > >> > (Usage-Job-1:null) (logid:) usage job complete > > > > >> > > > > > >> > > > > > >> > mysql> select id, pid, start_date ,end_date , success, > heartbeat > > > from > > > > >> > usage_job where success=0 order by start_date desc limit 100; > > > > >> > > > > > >> you should search for `where success is null` > > > > >> > > > > >> > > > > >> > Empty set (0.00 sec) > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > On Fri, Jun 18, 2021 at 11:01 PM Hean Seng > > > > wrote: > > > > >> > > > > > >> > > yes, I restarted the mgmt server, wait till can login web > > > > interface > > > > >> . > > > > >> > > then only restart usage . > > > > >> > > > > > > >> > > I did follwoig also : > > > > >> > > > > > > >> > > * stop usage server > > > > >> > > > > > > >> > > * remove the record from usage_job for the unprocessed job > > > > >> > > > > > > >> > > * start usage server again > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > Still same thing. Not sure where the Cloudstack keep the > Usage > > > > server > > > > >> > PID > > > > >> > > id , and keep getting wrong id. > > > > >> > > > > > > >> > > > > > > >> > > > > >
Re: Cloudstack Usage --- not owner
Seng > > > wrote: > > > >> > > > > >> > > yes, I restarted the mgmt server, wait till can login web > > > interface > > > >> . > > > >> > > then only restart usage . > > > >> > > > > > >> > > I did follwoig also : > > > >> > > > > > >> > > * stop usage server > > > >> > > > > > >> > > * remove the record from usage_job for the unprocessed job > > > >> > > > > > >> > > * start usage server again > > > >> > > > > > >> > > > > > >> > > > > > >> > > Still same thing. Not sure where the Cloudstack keep the Usage > > > server > > > >> > PID > > > >> > > id , and keep getting wrong id. > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > On Fri, Jun 18, 2021 at 4:31 PM Boris Stoyanov < > > > >> > > boris.stoya...@shapeblue.com> wrote: > > > >> > > > > > >> > >> Hi Hean Seng, > > > >> > >> > > > >> > >> Have you waited the management service to come up and then > > > restarted > > > >> > >> usage service? It might be some race condition? > > > >> > >> > > > >> > >> Bobby. > > > >> > >> > > > >> > >> From: Hean Seng > > > >> > >> Date: Friday, 18 June 2021, 7:31 > > > >> > >> To: users@cloudstack.apache.org > > > >> > >> Subject: Cloudstack Usage --- not owner > > > >> > >> Hi > > > >> > >> > > > >> > >> I have mysql hang, and restarted mysql, restarted cloudstack > > mgmt, > > > >> and > > > >> > >> usage. > > > >> > >> > > > >> > >> After that facing issue of following: > > > >> > >> > > > >> > >> duration is 120 minutes) > > > >> > >> > > > >> > >> 2021-06-18 04:25:49,656 DEBUG [cloud.usage.UsageManagerImpl] > > > >> > >> (Usage-HB-1:null) (logid:) Scheduling Usage job... > > > >> > >> > > > >> > >> 2021-06-18 04:25:49,657 INFO [cloud.usage.UsageManagerImpl] > > > >> > >> (Usage-Job-1:null) (logid:) starting usage job... > > > >> > >> > > > >> > >> 2021-06-18 04:25:49,669 DEBUG [cloud.usage.UsageManagerImpl] > > > >> > >> (Usage-Job-1:null) (logid:) Not owner of usage job, skipping... > > > >> > >> > > > >> > >> 2021-06-18 04:25:49,669 INFO [cloud.usage.UsageManagerImpl] > > > >> > >> (Usage-Job-1:null) (logid:) usage job complete > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> Seems usage pid is not updated to db . > > > >> > >> > > > >> > >> Andybody know how to fix this ? > > > >> > >> > > > >> > >> -- > > > >> > >> Regards, > > > >> > >> Hean Seng > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > > > > > >> > > -- > > > >> > > Regards, > > > >> > > Hean Seng > > > >> > > > > > >> > > > > >> > > > > >> > -- > > > >> > Regards, > > > >> > Hean Seng > > > >> > > > > >> > > > >> > > > >> -- > > > >> Daan > > > >> > > > > > > > > > > > > -- > > > > Regards, > > > > Hean Seng > > > > > > > > > > > > > -- > > > Regards, > > > Hean Seng > > > > > > > > > -- > > Daan > > > > > -- > Regards, > Hean Seng > -- Daan
Re: Cloudstack Usage --- not owner
Hi Daan, Thanks for reply. I deleted the null record, and manage to make it work again. The it will auto generate back the record with correct ID . However, I am facing the issue is the Usage Type 4, and 5 (Network sent and received) are not populated . Other Type like Type1 is always working. Fyi, I am using share network, and set the parameter vm.network.stats.interval=300 so that the VM statistic is updated . This issue not happen on all VM, but part of it. ( about 50%) . I tried look at the DB, and seem cannot find the clue what actually diff , in term of those updated and those is not. The record is appearing in usage_network_offering and seems good. Do you have any idea of this ? On Mon, Jun 21, 2021 at 3:19 PM Daan Hoogland wrote: > Sorry Hean, wasn't on line most of the weekend, Is it running now? > as you wrote this two days ago, it should already have run twice (assuming > you didn't change a lot) > > On Sat, Jun 19, 2021 at 8:41 PM Hean Seng wrote: > > > And it will insert another success is NULL to usage_job table , right ? > > > > i deleted the null record and restart usage servoce and see following > in > > usage log : > > > > 2021-06-19 18:37:52,912 INFO [cloud.usage.UsageManagerImpl] (main:null) > > (logid:) Implementation Version is 4.15.0.0 > > > > 2021-06-19 18:37:55,942 DEBUG [cloud.usage.UsageManagerImpl] (main:null) > > (logid:) Usage stats aggregation time zone: GMT > > > > 2021-06-19 18:37:55,952 DEBUG [cloud.usage.UsageManagerImpl] (main:null) > > (logid:) Execution Time: Sat Jun 19 00:15:00 UTC 2021 > > > > 2021-06-19 18:37:55,953 DEBUG [cloud.usage.UsageManagerImpl] (main:null) > > (logid:) Current Time: Sat Jun 19 18:37:55 UTC 2021 > > > > It seem execution time before current time ? Is that normal ? > > > > > > > > > > > > On Sun, Jun 20, 2021 at 2:39 AM Hean Seng wrote: > > > > > > > > So, I shall stop the Cloudstack-Usage services and delete all record > > where > > > success is NULL , and start back the usage service ? > > > > > > > > > On Sat, Jun 19, 2021 at 11:00 PM Daan Hoogland < > daan.hoogl...@gmail.com> > > > wrote: > > > > > >> Heanm, > > >> > > >> On Sat, Jun 19, 2021 at 5:27 AM Hean Seng wrote: > > >> > > >> > i have no unprocess job , but still have this issue: > > >> > > > >> > 2021-06-19 10:09:57,657 DEBUG [cloud.usage.UsageManagerImpl] > > >> > (Usage-HB-1:null) (logid:) Scheduling Usage job... > > >> > > > >> > 2021-06-19 10:09:57,657 INFO [cloud.usage.UsageManagerImpl] > > >> > (Usage-Job-1:null) (logid:) starting usage job... > > >> > > > >> > 2021-06-19 10:09:57,661 DEBUG [cloud.usage.UsageManagerImpl] > > >> > (Usage-Job-1:null) (logid:) Not owner of usage job, skipping... > > >> > > > >> this line ^^ means there is a unprocessed job owned by another > proccess > > >> ... > > >> > > >> > > >> > 2021-06-19 10:09:57,661 INFO [cloud.usage.UsageManagerImpl] > > >> > (Usage-Job-1:null) (logid:) usage job complete > > >> > > > >> > > > >> > mysql> select id, pid, start_date ,end_date , success, heartbeat > from > > >> > usage_job where success=0 order by start_date desc limit 100; > > >> > > > >> you should search for `where success is null` > > >> > > >> > > >> > Empty set (0.00 sec) > > >> > > > >> > > > >> > > > >> > > > >> > On Fri, Jun 18, 2021 at 11:01 PM Hean Seng > > wrote: > > >> > > > >> > > yes, I restarted the mgmt server, wait till can login web > > interface > > >> . > > >> > > then only restart usage . > > >> > > > > >> > > I did follwoig also : > > >> > > > > >> > > * stop usage server > > >> > > > > >> > > * remove the record from usage_job for the unprocessed job > > >> > > > > >> > > * start usage server again > > >> > > > > >> > > > > >> > > > > >> > > Still same thing. Not sure where the Cloudstack keep the Usage > > server > > >> > PID > > >> > > id , and keep getting wrong id. > > >> > > >
Re: Cloudstack Usage --- not owner
Sorry Hean, wasn't on line most of the weekend, Is it running now? as you wrote this two days ago, it should already have run twice (assuming you didn't change a lot) On Sat, Jun 19, 2021 at 8:41 PM Hean Seng wrote: > And it will insert another success is NULL to usage_job table , right ? > > i deleted the null record and restart usage servoce and see following in > usage log : > > 2021-06-19 18:37:52,912 INFO [cloud.usage.UsageManagerImpl] (main:null) > (logid:) Implementation Version is 4.15.0.0 > > 2021-06-19 18:37:55,942 DEBUG [cloud.usage.UsageManagerImpl] (main:null) > (logid:) Usage stats aggregation time zone: GMT > > 2021-06-19 18:37:55,952 DEBUG [cloud.usage.UsageManagerImpl] (main:null) > (logid:) Execution Time: Sat Jun 19 00:15:00 UTC 2021 > > 2021-06-19 18:37:55,953 DEBUG [cloud.usage.UsageManagerImpl] (main:null) > (logid:) Current Time: Sat Jun 19 18:37:55 UTC 2021 > > It seem execution time before current time ? Is that normal ? > > > > > > On Sun, Jun 20, 2021 at 2:39 AM Hean Seng wrote: > > > > > So, I shall stop the Cloudstack-Usage services and delete all record > where > > success is NULL , and start back the usage service ? > > > > > > On Sat, Jun 19, 2021 at 11:00 PM Daan Hoogland > > wrote: > > > >> Heanm, > >> > >> On Sat, Jun 19, 2021 at 5:27 AM Hean Seng wrote: > >> > >> > i have no unprocess job , but still have this issue: > >> > > >> > 2021-06-19 10:09:57,657 DEBUG [cloud.usage.UsageManagerImpl] > >> > (Usage-HB-1:null) (logid:) Scheduling Usage job... > >> > > >> > 2021-06-19 10:09:57,657 INFO [cloud.usage.UsageManagerImpl] > >> > (Usage-Job-1:null) (logid:) starting usage job... > >> > > >> > 2021-06-19 10:09:57,661 DEBUG [cloud.usage.UsageManagerImpl] > >> > (Usage-Job-1:null) (logid:) Not owner of usage job, skipping... > >> > > >> this line ^^ means there is a unprocessed job owned by another proccess > >> ... > >> > >> > >> > 2021-06-19 10:09:57,661 INFO [cloud.usage.UsageManagerImpl] > >> > (Usage-Job-1:null) (logid:) usage job complete > >> > > >> > > >> > mysql> select id, pid, start_date ,end_date , success, heartbeat from > >> > usage_job where success=0 order by start_date desc limit 100; > >> > > >> you should search for `where success is null` > >> > >> > >> > Empty set (0.00 sec) > >> > > >> > > >> > > >> > > >> > On Fri, Jun 18, 2021 at 11:01 PM Hean Seng > wrote: > >> > > >> > > yes, I restarted the mgmt server, wait till can login web > interface > >> . > >> > > then only restart usage . > >> > > > >> > > I did follwoig also : > >> > > > >> > > * stop usage server > >> > > > >> > > * remove the record from usage_job for the unprocessed job > >> > > > >> > > * start usage server again > >> > > > >> > > > >> > > > >> > > Still same thing. Not sure where the Cloudstack keep the Usage > server > >> > PID > >> > > id , and keep getting wrong id. > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > On Fri, Jun 18, 2021 at 4:31 PM Boris Stoyanov < > >> > > boris.stoya...@shapeblue.com> wrote: > >> > > > >> > >> Hi Hean Seng, > >> > >> > >> > >> Have you waited the management service to come up and then > restarted > >> > >> usage service? It might be some race condition? > >> > >> > >> > >> Bobby. > >> > >> > >> > >> From: Hean Seng > >> > >> Date: Friday, 18 June 2021, 7:31 > >> > >> To: users@cloudstack.apache.org > >> > >> Subject: Cloudstack Usage --- not owner > >> > >> Hi > >> > >> > >> > >> I have mysql hang, and restarted mysql, restarted cloudstack mgmt, > >> and > >> > >> usage. > >> > >> > >> > >> After that facing issue of following: > >> > >> > >> > >> duration is 120 minutes) > >> > >> > >> > >> 2021-06-18 04:25:49,656 DEBUG [cloud.usage.UsageManagerImpl] > >> > >> (Usage-HB-1:null) (logid:) Scheduling Usage job... > >> > >> > >> > >> 2021-06-18 04:25:49,657 INFO [cloud.usage.UsageManagerImpl] > >> > >> (Usage-Job-1:null) (logid:) starting usage job... > >> > >> > >> > >> 2021-06-18 04:25:49,669 DEBUG [cloud.usage.UsageManagerImpl] > >> > >> (Usage-Job-1:null) (logid:) Not owner of usage job, skipping... > >> > >> > >> > >> 2021-06-18 04:25:49,669 INFO [cloud.usage.UsageManagerImpl] > >> > >> (Usage-Job-1:null) (logid:) usage job complete > >> > >> > >> > >> > >> > >> > >> > >> Seems usage pid is not updated to db . > >> > >> > >> > >> Andybody know how to fix this ? > >> > >> > >> > >> -- > >> > >> Regards, > >> > >> Hean Seng > >> > >> > >> > >> > >> > >> > >> > >> > >> > > > >> > > -- > >> > > Regards, > >> > > Hean Seng > >> > > > >> > > >> > > >> > -- > >> > Regards, > >> > Hean Seng > >> > > >> > >> > >> -- > >> Daan > >> > > > > > > -- > > Regards, > > Hean Seng > > > > > -- > Regards, > Hean Seng > -- Daan
Re: Cloudstack Usage --- not owner
So, I shall stop the Cloudstack-Usage services and delete all record where success is NULL , and start back the usage service ? On Sat, Jun 19, 2021 at 11:00 PM Daan Hoogland wrote: > Heanm, > > On Sat, Jun 19, 2021 at 5:27 AM Hean Seng wrote: > > > i have no unprocess job , but still have this issue: > > > > 2021-06-19 10:09:57,657 DEBUG [cloud.usage.UsageManagerImpl] > > (Usage-HB-1:null) (logid:) Scheduling Usage job... > > > > 2021-06-19 10:09:57,657 INFO [cloud.usage.UsageManagerImpl] > > (Usage-Job-1:null) (logid:) starting usage job... > > > > 2021-06-19 10:09:57,661 DEBUG [cloud.usage.UsageManagerImpl] > > (Usage-Job-1:null) (logid:) Not owner of usage job, skipping... > > > this line ^^ means there is a unprocessed job owned by another proccess > ... > > > > 2021-06-19 10:09:57,661 INFO [cloud.usage.UsageManagerImpl] > > (Usage-Job-1:null) (logid:) usage job complete > > > > > > mysql> select id, pid, start_date ,end_date , success, heartbeat from > > usage_job where success=0 order by start_date desc limit 100; > > > you should search for `where success is null` > > > > Empty set (0.00 sec) > > > > > > > > > > On Fri, Jun 18, 2021 at 11:01 PM Hean Seng wrote: > > > > > yes, I restarted the mgmt server, wait till can login web interface . > > > then only restart usage . > > > > > > I did follwoig also : > > > > > > * stop usage server > > > > > > * remove the record from usage_job for the unprocessed job > > > > > > * start usage server again > > > > > > > > > > > > Still same thing. Not sure where the Cloudstack keep the Usage server > > PID > > > id , and keep getting wrong id. > > > > > > > > > > > > > > > > > > > > > On Fri, Jun 18, 2021 at 4:31 PM Boris Stoyanov < > > > boris.stoya...@shapeblue.com> wrote: > > > > > >> Hi Hean Seng, > > >> > > >> Have you waited the management service to come up and then restarted > > >> usage service? It might be some race condition? > > >> > > >> Bobby. > > >> > > >> From: Hean Seng > > >> Date: Friday, 18 June 2021, 7:31 > > >> To: users@cloudstack.apache.org > > >> Subject: Cloudstack Usage --- not owner > > >> Hi > > >> > > >> I have mysql hang, and restarted mysql, restarted cloudstack mgmt, and > > >> usage. > > >> > > >> After that facing issue of following: > > >> > > >> duration is 120 minutes) > > >> > > >> 2021-06-18 04:25:49,656 DEBUG [cloud.usage.UsageManagerImpl] > > >> (Usage-HB-1:null) (logid:) Scheduling Usage job... > > >> > > >> 2021-06-18 04:25:49,657 INFO [cloud.usage.UsageManagerImpl] > > >> (Usage-Job-1:null) (logid:) starting usage job... > > >> > > >> 2021-06-18 04:25:49,669 DEBUG [cloud.usage.UsageManagerImpl] > > >> (Usage-Job-1:null) (logid:) Not owner of usage job, skipping... > > >> > > >> 2021-06-18 04:25:49,669 INFO [cloud.usage.UsageManagerImpl] > > >> (Usage-Job-1:null) (logid:) usage job complete > > >> > > >> > > >> > > >> Seems usage pid is not updated to db . > > >> > > >> Andybody know how to fix this ? > > >> > > >> -- > > >> Regards, > > >> Hean Seng > > >> > > >> > > >> > > >> > > > > > > -- > > > Regards, > > > Hean Seng > > > > > > > > > -- > > Regards, > > Hean Seng > > > > > -- > Daan > -- Regards, Hean Seng
Re: Cloudstack Usage --- not owner
And it will insert another success is NULL to usage_job table , right ? i deleted the null record and restart usage servoce and see following in usage log : 2021-06-19 18:37:52,912 INFO [cloud.usage.UsageManagerImpl] (main:null) (logid:) Implementation Version is 4.15.0.0 2021-06-19 18:37:55,942 DEBUG [cloud.usage.UsageManagerImpl] (main:null) (logid:) Usage stats aggregation time zone: GMT 2021-06-19 18:37:55,952 DEBUG [cloud.usage.UsageManagerImpl] (main:null) (logid:) Execution Time: Sat Jun 19 00:15:00 UTC 2021 2021-06-19 18:37:55,953 DEBUG [cloud.usage.UsageManagerImpl] (main:null) (logid:) Current Time: Sat Jun 19 18:37:55 UTC 2021 It seem execution time before current time ? Is that normal ? On Sun, Jun 20, 2021 at 2:39 AM Hean Seng wrote: > > So, I shall stop the Cloudstack-Usage services and delete all record where > success is NULL , and start back the usage service ? > > > On Sat, Jun 19, 2021 at 11:00 PM Daan Hoogland > wrote: > >> Heanm, >> >> On Sat, Jun 19, 2021 at 5:27 AM Hean Seng wrote: >> >> > i have no unprocess job , but still have this issue: >> > >> > 2021-06-19 10:09:57,657 DEBUG [cloud.usage.UsageManagerImpl] >> > (Usage-HB-1:null) (logid:) Scheduling Usage job... >> > >> > 2021-06-19 10:09:57,657 INFO [cloud.usage.UsageManagerImpl] >> > (Usage-Job-1:null) (logid:) starting usage job... >> > >> > 2021-06-19 10:09:57,661 DEBUG [cloud.usage.UsageManagerImpl] >> > (Usage-Job-1:null) (logid:) Not owner of usage job, skipping... >> > >> this line ^^ means there is a unprocessed job owned by another proccess >> ... >> >> >> > 2021-06-19 10:09:57,661 INFO [cloud.usage.UsageManagerImpl] >> > (Usage-Job-1:null) (logid:) usage job complete >> > >> > >> > mysql> select id, pid, start_date ,end_date , success, heartbeat from >> > usage_job where success=0 order by start_date desc limit 100; >> > >> you should search for `where success is null` >> >> >> > Empty set (0.00 sec) >> > >> > >> > >> > >> > On Fri, Jun 18, 2021 at 11:01 PM Hean Seng wrote: >> > >> > > yes, I restarted the mgmt server, wait till can login web interface >> . >> > > then only restart usage . >> > > >> > > I did follwoig also : >> > > >> > > * stop usage server >> > > >> > > * remove the record from usage_job for the unprocessed job >> > > >> > > * start usage server again >> > > >> > > >> > > >> > > Still same thing. Not sure where the Cloudstack keep the Usage server >> > PID >> > > id , and keep getting wrong id. >> > > >> > > >> > > >> > > >> > > >> > > >> > > On Fri, Jun 18, 2021 at 4:31 PM Boris Stoyanov < >> > > boris.stoya...@shapeblue.com> wrote: >> > > >> > >> Hi Hean Seng, >> > >> >> > >> Have you waited the management service to come up and then restarted >> > >> usage service? It might be some race condition? >> > >> >> > >> Bobby. >> > >> >> > >> From: Hean Seng >> > >> Date: Friday, 18 June 2021, 7:31 >> > >> To: users@cloudstack.apache.org >> > >> Subject: Cloudstack Usage --- not owner >> > >> Hi >> > >> >> > >> I have mysql hang, and restarted mysql, restarted cloudstack mgmt, >> and >> > >> usage. >> > >> >> > >> After that facing issue of following: >> > >> >> > >> duration is 120 minutes) >> > >> >> > >> 2021-06-18 04:25:49,656 DEBUG [cloud.usage.UsageManagerImpl] >> > >> (Usage-HB-1:null) (logid:) Scheduling Usage job... >> > >> >> > >> 2021-06-18 04:25:49,657 INFO [cloud.usage.UsageManagerImpl] >> > >> (Usage-Job-1:null) (logid:) starting usage job... >> > >> >> > >> 2021-06-18 04:25:49,669 DEBUG [cloud.usage.UsageManagerImpl] >> > >> (Usage-Job-1:null) (logid:) Not owner of usage job, skipping... >> > >> >> > >> 2021-06-18 04:25:49,669 INFO [cloud.usage.UsageManagerImpl] >> > >> (Usage-Job-1:null) (logid:) usage job complete >> > >> >> > >> >> > >> >> > >> Seems usage pid is not updated to db . >> > >> >> > >> Andybody know how to fix this ? >> > >> >> > >> -- >> > >> Regards, >> > >> Hean Seng >> > >> >> > >> >> > >> >> > >> >> > > >> > > -- >> > > Regards, >> > > Hean Seng >> > > >> > >> > >> > -- >> > Regards, >> > Hean Seng >> > >> >> >> -- >> Daan >> > > > -- > Regards, > Hean Seng > -- Regards, Hean Seng
Re: Cloudstack Usage --- not owner
Heanm, On Sat, Jun 19, 2021 at 5:27 AM Hean Seng wrote: > i have no unprocess job , but still have this issue: > > 2021-06-19 10:09:57,657 DEBUG [cloud.usage.UsageManagerImpl] > (Usage-HB-1:null) (logid:) Scheduling Usage job... > > 2021-06-19 10:09:57,657 INFO [cloud.usage.UsageManagerImpl] > (Usage-Job-1:null) (logid:) starting usage job... > > 2021-06-19 10:09:57,661 DEBUG [cloud.usage.UsageManagerImpl] > (Usage-Job-1:null) (logid:) Not owner of usage job, skipping... > this line ^^ means there is a unprocessed job owned by another proccess ... > 2021-06-19 10:09:57,661 INFO [cloud.usage.UsageManagerImpl] > (Usage-Job-1:null) (logid:) usage job complete > > > mysql> select id, pid, start_date ,end_date , success, heartbeat from > usage_job where success=0 order by start_date desc limit 100; > you should search for `where success is null` > Empty set (0.00 sec) > > > > > On Fri, Jun 18, 2021 at 11:01 PM Hean Seng wrote: > > > yes, I restarted the mgmt server, wait till can login web interface . > > then only restart usage . > > > > I did follwoig also : > > > > * stop usage server > > > > * remove the record from usage_job for the unprocessed job > > > > * start usage server again > > > > > > > > Still same thing. Not sure where the Cloudstack keep the Usage server > PID > > id , and keep getting wrong id. > > > > > > > > > > > > > > On Fri, Jun 18, 2021 at 4:31 PM Boris Stoyanov < > > boris.stoya...@shapeblue.com> wrote: > > > >> Hi Hean Seng, > >> > >> Have you waited the management service to come up and then restarted > >> usage service? It might be some race condition? > >> > >> Bobby. > >> > >> From: Hean Seng > >> Date: Friday, 18 June 2021, 7:31 > >> To: users@cloudstack.apache.org > >> Subject: Cloudstack Usage --- not owner > >> Hi > >> > >> I have mysql hang, and restarted mysql, restarted cloudstack mgmt, and > >> usage. > >> > >> After that facing issue of following: > >> > >> duration is 120 minutes) > >> > >> 2021-06-18 04:25:49,656 DEBUG [cloud.usage.UsageManagerImpl] > >> (Usage-HB-1:null) (logid:) Scheduling Usage job... > >> > >> 2021-06-18 04:25:49,657 INFO [cloud.usage.UsageManagerImpl] > >> (Usage-Job-1:null) (logid:) starting usage job... > >> > >> 2021-06-18 04:25:49,669 DEBUG [cloud.usage.UsageManagerImpl] > >> (Usage-Job-1:null) (logid:) Not owner of usage job, skipping... > >> > >> 2021-06-18 04:25:49,669 INFO [cloud.usage.UsageManagerImpl] > >> (Usage-Job-1:null) (logid:) usage job complete > >> > >> > >> > >> Seems usage pid is not updated to db . > >> > >> Andybody know how to fix this ? > >> > >> -- > >> Regards, > >> Hean Seng > >> > >> > >> > >> > > > > -- > > Regards, > > Hean Seng > > > > > -- > Regards, > Hean Seng > -- Daan
Re: Cloudstack Usage --- not owner
i have no unprocess job , but still have this issue: 2021-06-19 10:09:57,657 DEBUG [cloud.usage.UsageManagerImpl] (Usage-HB-1:null) (logid:) Scheduling Usage job... 2021-06-19 10:09:57,657 INFO [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) (logid:) starting usage job... 2021-06-19 10:09:57,661 DEBUG [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) (logid:) Not owner of usage job, skipping... 2021-06-19 10:09:57,661 INFO [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) (logid:) usage job complete mysql> select id, pid, start_date ,end_date , success, heartbeat from usage_job where success=0 order by start_date desc limit 100; Empty set (0.00 sec) On Fri, Jun 18, 2021 at 11:01 PM Hean Seng wrote: > yes, I restarted the mgmt server, wait till can login web interface . > then only restart usage . > > I did follwoig also : > > * stop usage server > > * remove the record from usage_job for the unprocessed job > > * start usage server again > > > > Still same thing. Not sure where the Cloudstack keep the Usage server PID > id , and keep getting wrong id. > > > > > > > On Fri, Jun 18, 2021 at 4:31 PM Boris Stoyanov < > boris.stoya...@shapeblue.com> wrote: > >> Hi Hean Seng, >> >> Have you waited the management service to come up and then restarted >> usage service? It might be some race condition? >> >> Bobby. >> >> From: Hean Seng >> Date: Friday, 18 June 2021, 7:31 >> To: users@cloudstack.apache.org >> Subject: Cloudstack Usage --- not owner >> Hi >> >> I have mysql hang, and restarted mysql, restarted cloudstack mgmt, and >> usage. >> >> After that facing issue of following: >> >> duration is 120 minutes) >> >> 2021-06-18 04:25:49,656 DEBUG [cloud.usage.UsageManagerImpl] >> (Usage-HB-1:null) (logid:) Scheduling Usage job... >> >> 2021-06-18 04:25:49,657 INFO [cloud.usage.UsageManagerImpl] >> (Usage-Job-1:null) (logid:) starting usage job... >> >> 2021-06-18 04:25:49,669 DEBUG [cloud.usage.UsageManagerImpl] >> (Usage-Job-1:null) (logid:) Not owner of usage job, skipping... >> >> 2021-06-18 04:25:49,669 INFO [cloud.usage.UsageManagerImpl] >> (Usage-Job-1:null) (logid:) usage job complete >> >> >> >> Seems usage pid is not updated to db . >> >> Andybody know how to fix this ? >> >> -- >> Regards, >> Hean Seng >> >> >> >> > > -- > Regards, > Hean Seng > -- Regards, Hean Seng
Re: Cloudstack Usage --- not owner
yes, I restarted the mgmt server, wait till can login web interface . then only restart usage . I did follwoig also : * stop usage server * remove the record from usage_job for the unprocessed job * start usage server again Still same thing. Not sure where the Cloudstack keep the Usage server PID id , and keep getting wrong id. On Fri, Jun 18, 2021 at 4:31 PM Boris Stoyanov wrote: > Hi Hean Seng, > > Have you waited the management service to come up and then restarted usage > service? It might be some race condition? > > Bobby. > > From: Hean Seng > Date: Friday, 18 June 2021, 7:31 > To: users@cloudstack.apache.org > Subject: Cloudstack Usage --- not owner > Hi > > I have mysql hang, and restarted mysql, restarted cloudstack mgmt, and > usage. > > After that facing issue of following: > > duration is 120 minutes) > > 2021-06-18 04:25:49,656 DEBUG [cloud.usage.UsageManagerImpl] > (Usage-HB-1:null) (logid:) Scheduling Usage job... > > 2021-06-18 04:25:49,657 INFO [cloud.usage.UsageManagerImpl] > (Usage-Job-1:null) (logid:) starting usage job... > > 2021-06-18 04:25:49,669 DEBUG [cloud.usage.UsageManagerImpl] > (Usage-Job-1:null) (logid:) Not owner of usage job, skipping... > > 2021-06-18 04:25:49,669 INFO [cloud.usage.UsageManagerImpl] > (Usage-Job-1:null) (logid:) usage job complete > > > > Seems usage pid is not updated to db . > > Andybody know how to fix this ? > > -- > Regards, > Hean Seng > > > > -- Regards, Hean Seng
Re: Cloudstack Usage --- not owner
Hi Hean Seng, Have you waited the management service to come up and then restarted usage service? It might be some race condition? Bobby. From: Hean Seng Date: Friday, 18 June 2021, 7:31 To: users@cloudstack.apache.org Subject: Cloudstack Usage --- not owner Hi I have mysql hang, and restarted mysql, restarted cloudstack mgmt, and usage. After that facing issue of following: duration is 120 minutes) 2021-06-18 04:25:49,656 DEBUG [cloud.usage.UsageManagerImpl] (Usage-HB-1:null) (logid:) Scheduling Usage job... 2021-06-18 04:25:49,657 INFO [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) (logid:) starting usage job... 2021-06-18 04:25:49,669 DEBUG [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) (logid:) Not owner of usage job, skipping... 2021-06-18 04:25:49,669 INFO [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) (logid:) usage job complete Seems usage pid is not updated to db . Andybody know how to fix this ? -- Regards, Hean Seng
Re: Cloudstack Usage --- not owner
Haen, This is a problem on exit. When stopping the usage server doesn't always clean its job. It is a problem but a redesign of the usage server in general is needed, especially for large environments/DBs. A workaround is after stopping the usage server, delete the unprocessed job with the old PID in the DB. You can then start the usage server and it should create its own record and process it. regards, On Fri, Jun 18, 2021 at 6:31 AM Hean Seng wrote: > Hi > > I have mysql hang, and restarted mysql, restarted cloudstack mgmt, and > usage. > > After that facing issue of following: > > duration is 120 minutes) > > 2021-06-18 04:25:49,656 DEBUG [cloud.usage.UsageManagerImpl] > (Usage-HB-1:null) (logid:) Scheduling Usage job... > > 2021-06-18 04:25:49,657 INFO [cloud.usage.UsageManagerImpl] > (Usage-Job-1:null) (logid:) starting usage job... > > 2021-06-18 04:25:49,669 DEBUG [cloud.usage.UsageManagerImpl] > (Usage-Job-1:null) (logid:) Not owner of usage job, skipping... > > 2021-06-18 04:25:49,669 INFO [cloud.usage.UsageManagerImpl] > (Usage-Job-1:null) (logid:) usage job complete > > > > Seems usage pid is not updated to db . > > Andybody know how to fix this ? > > -- > Regards, > Hean Seng > -- Daan
Cloudstack Usage --- not owner
Hi I have mysql hang, and restarted mysql, restarted cloudstack mgmt, and usage. After that facing issue of following: duration is 120 minutes) 2021-06-18 04:25:49,656 DEBUG [cloud.usage.UsageManagerImpl] (Usage-HB-1:null) (logid:) Scheduling Usage job... 2021-06-18 04:25:49,657 INFO [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) (logid:) starting usage job... 2021-06-18 04:25:49,669 DEBUG [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) (logid:) Not owner of usage job, skipping... 2021-06-18 04:25:49,669 INFO [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) (logid:) usage job complete Seems usage pid is not updated to db . Andybody know how to fix this ? -- Regards, Hean Seng
