Hi Nux, I’d like to give another perspective here…
Monitoring ACS entails the following… Reading ACS DB (yes, in many cases you will need to/find it simpler) Calling API/cloudmonkey Calling custom scripts which will run on Hosts and monitoring servers. SNMP Calls Agent parameters You will need to use all these methods. Limiting yourself to the API alone will not solve all your use cases(IMHO). Once you are ready with the above, the actual monitoring tool you use is totally your choice. Cheers, Shiv > On 08-Mar-2022, at 20:45, Ivet Petrova <ivet.petr...@shapeblue.com> wrote: > > Hi, > > To help a bit Nux I designed a short survey. Pls, take a moment to tick a few > boxes on what monitoring solutions you are using. > > Will take you not more than 15 seconds: https://forms.gle/V17GQziAFn8ZdoVy6 > <https://forms.gle/V17GQziAFn8ZdoVy6> > > > Kind regards, > > > > > On 7 Mar 2022, at 17:47, Vladimir Dombrovski <vladimir.dombrov...@bso.co > <mailto:vladimir.dombrov...@bso.co><mailto:vladimir.dombrov...@bso.co > <mailto:vladimir.dombrov...@bso.co>>> wrote: > > Hello Nux, > > We've had issues with the following APIs: > > - https://cloudstack.apache.org/api/apidocs-4.15/apis/listCapacity.html > -> global dashboard view, this is where we've had 132% CPU usage + > confusion about frequency / cores > - > https://cloudstack.apache.org/api/apidocs-4.15/apis/listVirtualMachinesMetrics.html > -> per VM metric view, this is where we've had inconsistent / wrong > memory usage metrics > > Currently, we are enriching our metadata using the following nested calls: > > - https://cloudstack.apache.org/api/apidocs-4.15/apis/listAccounts.html > - https://cloudstack.apache.org/api/apidocs-4.15/apis/listProjects.html > And > - https://cloudstack.apache.org/api/apidocs-4.15/apis/listVirtualMachines.html > > This gives us a per account and a per project VM list, which we are > able to correlate with the metrics gathered by our cgroups collector. > > Note: we're using ACS 4.15.2 in production, and currently testing ACS > 4.16.0, with the goal of upgrading to it by the end of the year. > > Regards, > > Vladimir > > > On Mon, 7 Mar 2022 at 15:08, Nux <n...@li.nux.ro > <mailto:n...@li.nux.ro><mailto:n...@li.nux.ro <mailto:n...@li.nux.ro>>> wrote: > > Hi Vladimir, > > Thanks for the elaborate reply. > Can you let me know which api calls you used to determine VM stats, > especially the inaccurate ones? > > Regards > > > --- > Nux! > www.nux.ro <http://www.nux.ro/><http://www.nux.ro <http://www.nux.ro/>> > > On 2022-03-07 13:56, Vladimir Dombrovski wrote: > Hello all, > > After having tried to collect the correct metrics from the API, we had > to resort to a lower level set of collectors, as some of the metrics > (e.g. per VM resource usage) were incorrect. > > Our current setup is netdata that uses the cgroups collector to gather > from KVM/QEMU. We then match the VM names with metadata which we are > fetching directly from the API to enrich the metrics with labels. > We're then storing everything inside Prometheus, and exposing the > graphs via a separate Grafana. > > Globally, the feedback we've had from our end-users is that after > seeing incorrect metrics (mostly in used RAM counters) they got really > confused about the actual state of their project. It is really > unsettling to see a VM configured to have multiple GB of RAM to use > 100% of it immediately at boot (which is incorrect, because the > incorrect underlying counter was used). We've also had other instances > where the CPU usage went above 100% which looks really confusing. > Finally, users were sceptical as to why there are two separate > measurements for CPU, one being the total frequency (whatever that is > supposed to mean), and the other the number of cores used. All these > encounters really hurt when advocating it as a trustworthy system to > our users. > > Now, we've tried some workarounds, for example, we tried compensating > for the incorrect counters by enabling memory ballooning, which > somewhat helped with the correct values but brought along its own set > of issues. In the future, we are planning to do a lot more tests on > the metrics, by comparing what Cloudstack gives us with what we > actually see inside our systems. We hope that we'll eventually find > the correct configuration elements, and maybe flag some issues if we > happen to find any. > > To sum up, we believe that the current implementation of metrics isn't > nearly accurate enough (at least out of the box) to be shown to all > users as a landing page. We also completely get it that Cloudstack is > a complex and complete ecosystem that has ties and dependencies to > many sub-components, making it difficult to try and test all the > possible use-cases. We hope to reach a point soon where we have used > and understand the system well enough to be able to contribute in a > meaningful way. > > Regards, > > Vladimir > > On Mon, 7 Mar 2022 at 12:02, Ivet Petrova <ivet.petr...@shapeblue.com> > wrote: > Maybe this talk from the last CloudStack Collaboration Conference can > be useful: https://www.youtube.com/watch?v=m8mYdWHoxLY > > > > Kind regards, > > > > > On 7 Mar 2022, at 12:10, benoit lair > <kurushi4...@gmail.com<mailto:kurushi4...@gmail.com>> wrote: > > Hi, > > We are now using Centreon with a custom autodeclaration feature done > with > our own templates (now acs 4.16 in production) > If we can use something more out of the box i would enjoy to change it > We used to use Zenoss for our ACS 4.3 which has a plugin specific to > Cloudstack, but it is no more maintained > > Regards, Benoit > > Le dim. 6 mars 2022 à 16:40, Paul Angus > <pau...@apache.org<mailto:pau...@apache.org>> a écrit : > > Hi Nux, > > At Ticketmaster we use the Prometheus exporter. We about to work on > adding > more detail to what's exported wrt to VMs, as it very infrastructure > focused > out-of-the-box. > > > > Kind regards > > Paul Angus > > -----Original Message----- > From: Nux <n...@li.nux.ro<mailto:n...@li.nux.ro>> > Sent: 02 March 2022 10:56 > To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org>; > d...@cloudstack.apache.org<mailto:d...@cloudstack.apache.org> > Subject: Re: How are you monitoring Cloudstack? > > Hi! > > Another nudge on the $subject in case people missed this. > > If you have a functioning way of monitoring Cloudstack & co in your > organisation I'd like to hear about it. > It doesn't have to be anything exotic, so don't be shy as long as we > have > anything to talk about. > > Thanks :) > > > On 2022-02-21 14:38, Nux! wrote: > Hi folks, > > If anyone cares to share (on or off list) with me a few words about > how they are monitoring Cloudstack and related infrastructure that'd > be lovely. > I'm trying to find out what are the choices currently and how we can > improve the overall experience. > > Don't be shy! > > Cheers > > > > > -- > *CONFIDENTIALITY AND DISCLAIMER NOTICE: * > This email is intended only for > the person to whom it is addressed and/or otherwise authorized > personnel. > The information contained herein and attached is confidential. If you > are > not the intended recipient, please be advised that viewing this message > and > any attachments, as well as copying, forwarding, printing, and > disseminating any information related to this email is prohibited, and > that > you should not take any action based on the content of this email > and/or > its attachments. If you received this message in error, please contact > the > sender and destroy all copies of this email and any attachment. Please > note > that the views and opinions expressed herein are solely those of the > author > and do not necessarily reflect those of the company. While antivirus > protection tools have been employed, you should check this email and > attachments for the presence of viruses. No warranties or assurances > are > made in relation to the safety and content of this email and > attachments. > The Company accepts no liability for any damage caused by any virus > transmitted by or contained in this email and attachments. No liability > is > accepted for any consequences arising from this email. > > > *AVIS DE > CONFIDENTIALITÉ ET DE NON RESPONSABILITE* : > Ce courriel, ainsi que toute > pièce jointe, est confidentiel et peut être protégé par le secret > professionnel. Si vous n’en êtes pas le destinataire visé, veuillez en > aviser l’expéditeur immédiatement et le supprimer. Vous ne devez pas le > copier, ni l’utiliser à quelque fin que ce soit, ni divulguer son > contenu à > qui que ce soit. BSO se réserve le droit de contrôler toute > transmission > qui passe par son réseau. Veuillez noter que les opinions exprimées > dans > cet e-mail sont uniquement celles de l'auteur et ne reflètent pas > nécessairement celles de la société. Bien que des outils de protection > antivirus aient été utilisés, vous devez vérifier cet e-mail et les > pièces > jointes pour toute présence de virus. Aucune garantie ou assurance > n'est > donnée concernant la sécurité et le contenu de cet e-mail et de ses > pièces > jointes. La Société décline toute responsabilité pour tout dommage > causé > par tout virus transmis par ou contenu dans cet e-mail et ses pièces > jointes. Aucune responsabilité n'est acceptée pour les conséquences > découlant de cet e-mail. > > -- > *CONFIDENTIALITY AND DISCLAIMER NOTICE: * > This email is intended only for > the person to whom it is addressed and/or otherwise authorized personnel. > The information contained herein and attached is confidential. If you are > not the intended recipient, please be advised that viewing this message and > any attachments, as well as copying, forwarding, printing, and > disseminating any information related to this email is prohibited, and that > you should not take any action based on the content of this email and/or > its attachments. If you received this message in error, please contact the > sender and destroy all copies of this email and any attachment. Please note > that the views and opinions expressed herein are solely those of the author > and do not necessarily reflect those of the company. While antivirus > protection tools have been employed, you should check this email and > attachments for the presence of viruses. No warranties or assurances are > made in relation to the safety and content of this email and attachments. > The Company accepts no liability for any damage caused by any virus > transmitted by or contained in this email and attachments. No liability is > accepted for any consequences arising from this email. > > > *AVIS DE > CONFIDENTIALITÉ ET DE NON RESPONSABILITE* : > Ce courriel, ainsi que toute > pièce jointe, est confidentiel et peut être protégé par le secret > professionnel. Si vous n’en êtes pas le destinataire visé, veuillez en > aviser l’expéditeur immédiatement et le supprimer. Vous ne devez pas le > copier, ni l’utiliser à quelque fin que ce soit, ni divulguer son contenu à > qui que ce soit. BSO se réserve le droit de contrôler toute transmission > qui passe par son réseau. Veuillez noter que les opinions exprimées dans > cet e-mail sont uniquement celles de l'auteur et ne reflètent pas > nécessairement celles de la société. Bien que des outils de protection > antivirus aient été utilisés, vous devez vérifier cet e-mail et les pièces > jointes pour toute présence de virus. Aucune garantie ou assurance n'est > donnée concernant la sécurité et le contenu de cet e-mail et de ses pièces > jointes. La Société décline toute responsabilité pour tout dommage causé > par tout virus transmis par ou contenu dans cet e-mail et ses pièces > jointes. Aucune responsabilité n'est acceptée pour les conséquences > découlant de cet e-mail. -- This message is intended only for the use of the individual or entity to which it is addressed and may contain confidential and/or privileged information. If you are not the intended recipient, please delete the original message and any copy of it from your computer system. You are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited unless proper authorization has been obtained for such action. If you have received this communication in error, please notify the sender immediately. Although IndiQus attempts to sweep e-mail and attachments for viruses, it does not guarantee that both are virus-free and accepts no liability for any damage sustained as a result of viruses.
