Re: CloudStack Logging
On Thu, Apr 21, 2016 at 9:49 AM, Paul Anguswrote: > Hey Daan, > > I understand from a purists point of view, but to me messages like: > > INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-113:ctx-56d9f9f2) > Scan hung worker VM to recycle > > Are part of the inner workings of cloudstack, and only a developer is > going to do anything with this information. > Unless we can add another category like INFO-INTERNAL between DEBUG and > INFO, a message like this is probably technically INFO, from anyone's > practical point of view, its DEBUG. > There is nothing purist about it. ACS is freeing resources which is a process taking resources. It is information important to an operator, maybe not every operator has a process using every bit of information. That doesn't mean it is not important. On one hand, you can always filter it in your monitoring or searching. On the other hand, if it doesn't mean anything to you it might mean that the phrasing is bad and should improve. -- Daan
RE: CloudStack Logging
Hi Frank, 'thinking out loud' - ideally I'd like to see this failure (and it reasons) surfaced to the admins outside of the logs. Off the top of my head - a (helpful) detailed event/alert notification. More helpful info in the message (ie not 'host 24') *should* be pretty simple. I'll have a look and ask one of our developers to help me with the actual code, once I've tracked it down. Kind regards, Paul Angus Regards, Paul Angus paul.an...@shapeblue.com www.shapeblue.com 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue -Original Message- From: Frank Louwers [mailto:fr...@openminds.be] Sent: 21 April 2016 08:14 To: users@cloudstack.apache.org Subject: Re: CloudStack Logging Hi Paul, I’d love to see improvement in that area! Especially for my operational admins, the biggest issue we face is this: - Someone wants to deploy a new VM, and it fails. - Currently, it’s very hard to figure out exactly why. The best they get is “Capacity Planner failure”. - They come to me :) I think these failures are quite an easy use-case to improve logging on. If a VM can’t be started because of a “capacity” issue, it would we good to log (in a non-debug, non-info log): - Which hosts were considered for CPU/Mem (based on zone, tags etc) (list them by name pref, not by “host 24” which means nothing to them as that number isn’t anywhere in the CS UI) - Which of those considered hosts have the capacity needed (and for those that are excluded, weather they are excluded for RAM or CPU reasons) - Exact same for Storage Pools This would solve over 95% of my “Frank, I can’t find out why this VM won’t deploy” logging issues. Regards, Frank > On 20 Apr 2016, at 22:05, Paul Angus <paul.an...@shapeblue.com> wrote: > > Hi, > > I don't think that it's a question of importance. INFO vs DEBUG should be > telling you different things. ERROR and WARN are also quite different. > In general I'm targeting the cloud operational admins, the people who need to > know the health of their cloud and deal with issues as they're constantly > reading the log. > > However, I'm not proposing to add or remove any messages, just revisit the > categorisation. If the 'operator' has their logging on debug they'd actually > see the same messages. The idea is to make turning the logging down to INFO > feasible. > > You'd turn the logging back up to pick up code issues rather than just > operational issues. > > Some VERY simplified definitions might be: > > DEBUG: inner workings of CloudStack that only a developer can 'understand' > INFO: actions that CloudStack is performing or information analysis (host 5 > is full so not going to use it). > WARN: this isn't good > ERROR: something just didn't work. > > > Grabbing a couple of examples > > these should be debug - I can't 'do' anything with this information. > > INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > (AsyncJobMgr-Heartbeat-1:ctx-4119d5bc) Begin cleanup expired > async-jobs INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > (AsyncJobMgr-Heartbeat-1:ctx-4119d5bc) End cleanup expired async-jobs > INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-2:ctx-3209f65c) > Scan hung worker VM to recycle INFO [c.c.h.v.r.VmwareResource] > (DirectAgentCronJob-113:ctx-56d9f9f2) Scan hung worker VM to recycle > > Whereas this should be WARN. I wouldn't want to lose this by switching > off DEBUG > > DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-91226be7) > Detected management node left, id:2, nodeIP:10.2.0.6 DEBUG > [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-0bccd078) Detected > management node left, id:2, nodeIP:10.2.0.6 > > > > > Kind regards, > > Paul Angus > > Regards, > > Paul Angus > > paul.an...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue > > -Original Message- > From: Chaz PC [mailto:dreeems4e...@hotmail.com] > Sent: 20 April 2016 11:32 > To: users@cloudstack.apache.org > Subject: RE: CloudStack Logging > > Hello, > I have a question what is the criteria that you are going to follow to decide > the log record importance. Who is the user that you will design the logging > process for. Is it for finding issues with the installation or is it for > security breaches or auditing. > These questions are good to put in mind when designing the logging process. > > > Sent from my Samsung Galaxy smartphone. > Original message > From: Paul Angus <paul.an...@shapeblue.com> > Date: 20/04/2016 1:01 PM (GMT+04:00) To: > d...@cloudstack.apache.org, users@cloudstack.apache.org > Subject: RE: CloudStack Logging Hi > Simon, > > My gut says that it's probably not wor
RE: CloudStack Logging
Hey Daan, I understand from a purists point of view, but to me messages like: INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-113:ctx-56d9f9f2) Scan hung worker VM to recycle Are part of the inner workings of cloudstack, and only a developer is going to do anything with this information. Unless we can add another category like INFO-INTERNAL between DEBUG and INFO, a message like this is probably technically INFO, from anyone's practical point of view, its DEBUG. Kind regards, Paul Angus Regards, Paul Angus paul.an...@shapeblue.com www.shapeblue.com 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue -Original Message- From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] Sent: 20 April 2016 23:12 To: users@cloudstack.apache.org Subject: Re: CloudStack Logging Paul, Your classification of levels seems right. I wouldn't say WARN means it isn't good but just that it might be given an objective the user might have. There is also FATAL, ERROR would be recoverable while FATAL means the system can no longer be trusted to do it's job right. The INFO examples you are giving are indeed INFO as far as I'm concerned and not debug; ACS is reporting what it is doing on a pretty high level. But we can discuss about it and even after we give proper levels to messages we can still decide to set levels for certain parts of the sys higher or lower then for others. On Wed, Apr 20, 2016 at 10:05 PM, Paul Angus <paul.an...@shapeblue.com> wrote: > Hi, > > I don't think that it's a question of importance. INFO vs DEBUG should > be telling you different things. ERROR and WARN are also quite different. > In general I'm targeting the cloud operational admins, the people who > need to know the health of their cloud and deal with issues as they're > constantly reading the log. > > However, I'm not proposing to add or remove any messages, just revisit > the categorisation. If the 'operator' has their logging on debug > they'd actually see the same messages. The idea is to make turning the > logging down to INFO feasible. > > You'd turn the logging back up to pick up code issues rather than just > operational issues. > > Some VERY simplified definitions might be: > > DEBUG: inner workings of CloudStack that only a developer can 'understand' > INFO: actions that CloudStack is performing or information analysis > (host > 5 is full so not going to use it). > WARN: this isn't good > ERROR: something just didn't work. > > > Grabbing a couple of examples > > these should be debug - I can't 'do' anything with this information. > > INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > (AsyncJobMgr-Heartbeat-1:ctx-4119d5bc) Begin cleanup expired > async-jobs INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > (AsyncJobMgr-Heartbeat-1:ctx-4119d5bc) End cleanup expired async-jobs > INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-2:ctx-3209f65c) > Scan hung worker VM to recycle INFO [c.c.h.v.r.VmwareResource] > (DirectAgentCronJob-113:ctx-56d9f9f2) > Scan hung worker VM to recycle > > Whereas this should be WARN. I wouldn't want to lose this by switching > off DEBUG > > DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-91226be7) > Detected management node left, id:2, nodeIP:10.2.0.6 DEBUG > [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-0bccd078) > Detected management node left, id:2, nodeIP:10.2.0.6 > > > > > Kind regards, > > Paul Angus > > Regards, > > Paul Angus > > paul.an...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue > > -Original Message- > From: Chaz PC [mailto:dreeems4e...@hotmail.com] > Sent: 20 April 2016 11:32 > To: users@cloudstack.apache.org > Subject: RE: CloudStack Logging > > Hello, > I have a question what is the criteria that you are going to follow to > decide the log record importance. Who is the user that you will design > the logging process for. Is it for finding issues with the > installation or is it for security breaches or auditing. > These questions are good to put in mind when designing the logging process. > > > Sent from my Samsung Galaxy smartphone. > Original message > From: Paul Angus <paul.an...@shapeblue.com> > Date: 20/04/2016 1:01 PM (GMT+04:00) To: > d...@cloudstack.apache.org, users@cloudstack.apache.org > Subject: RE: CloudStack Logging Hi > Simon, > > My gut says that it's probably not worth going back before 4.3. There > have been large changes since then but I think that it's better to get > as much data as possible, limiting to 4.6+ might not give a > particularly large install base. > > > Kind regards, > > Paul Angus > > Regards, > > Paul Angus >
Re: CloudStack Logging
Hi Paul, I’d love to see improvement in that area! Especially for my operational admins, the biggest issue we face is this: - Someone wants to deploy a new VM, and it fails. - Currently, it’s very hard to figure out exactly why. The best they get is “Capacity Planner failure”. - They come to me :) I think these failures are quite an easy use-case to improve logging on. If a VM can’t be started because of a “capacity” issue, it would we good to log (in a non-debug, non-info log): - Which hosts were considered for CPU/Mem (based on zone, tags etc) (list them by name pref, not by “host 24” which means nothing to them as that number isn’t anywhere in the CS UI) - Which of those considered hosts have the capacity needed (and for those that are excluded, weather they are excluded for RAM or CPU reasons) - Exact same for Storage Pools This would solve over 95% of my “Frank, I can’t find out why this VM won’t deploy” logging issues. Regards, Frank > On 20 Apr 2016, at 22:05, Paul Angus <paul.an...@shapeblue.com> wrote: > > Hi, > > I don't think that it's a question of importance. INFO vs DEBUG should be > telling you different things. ERROR and WARN are also quite different. > In general I'm targeting the cloud operational admins, the people who need to > know the health of their cloud and deal with issues as they're constantly > reading the log. > > However, I'm not proposing to add or remove any messages, just revisit the > categorisation. If the 'operator' has their logging on debug they'd actually > see the same messages. The idea is to make turning the logging down to INFO > feasible. > > You'd turn the logging back up to pick up code issues rather than just > operational issues. > > Some VERY simplified definitions might be: > > DEBUG: inner workings of CloudStack that only a developer can 'understand' > INFO: actions that CloudStack is performing or information analysis (host 5 > is full so not going to use it). > WARN: this isn't good > ERROR: something just didn't work. > > > Grabbing a couple of examples > > these should be debug - I can't 'do' anything with this information. > > INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > (AsyncJobMgr-Heartbeat-1:ctx-4119d5bc) Begin cleanup expired async-jobs > INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > (AsyncJobMgr-Heartbeat-1:ctx-4119d5bc) End cleanup expired async-jobs > INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-2:ctx-3209f65c) Scan > hung worker VM to recycle > INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-113:ctx-56d9f9f2) Scan > hung worker VM to recycle > > Whereas this should be WARN. I wouldn't want to lose this by switching off > DEBUG > > DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-91226be7) Detected > management node left, id:2, nodeIP:10.2.0.6 > DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-0bccd078) Detected > management node left, id:2, nodeIP:10.2.0.6 > > > > > Kind regards, > > Paul Angus > > Regards, > > Paul Angus > > paul.an...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK > @shapeblue > > -Original Message- > From: Chaz PC [mailto:dreeems4e...@hotmail.com] > Sent: 20 April 2016 11:32 > To: users@cloudstack.apache.org > Subject: RE: CloudStack Logging > > Hello, > I have a question what is the criteria that you are going to follow to decide > the log record importance. Who is the user that you will design the logging > process for. Is it for finding issues with the installation or is it for > security breaches or auditing. > These questions are good to put in mind when designing the logging process. > > > Sent from my Samsung Galaxy smartphone. > Original message > From: Paul Angus <paul.an...@shapeblue.com> > Date: 20/04/2016 1:01 PM (GMT+04:00) To: > d...@cloudstack.apache.org, users@cloudstack.apache.org Subject: > RE: CloudStack Logging Hi Simon, > > My gut says that it's probably not worth going back before 4.3. There have > been large changes since then but I think that it's better to get as much > data as possible, limiting to 4.6+ might not give a particularly large > install base. > > > Kind regards, > > Paul Angus > > Regards, > > Paul Angus > > paul.an...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue > > -Original Message- > From: Simon Weller [mailto:swel...@ena.com] > Sent: 19 April 2016 21:51 > To: users@cloudstack.apache.org; d...@cloudstack.apache.org > Subject: Re: CloudStack Logging > > Paul, > > Are you wanting for focus on logs o
Re: CloudStack Logging
Paul, Your classification of levels seems right. I wouldn't say WARN means it isn't good but just that it might be given an objective the user might have. There is also FATAL, ERROR would be recoverable while FATAL means the system can no longer be trusted to do it's job right. The INFO examples you are giving are indeed INFO as far as I'm concerned and not debug; ACS is reporting what it is doing on a pretty high level. But we can discuss about it and even after we give proper levels to messages we can still decide to set levels for certain parts of the sys higher or lower then for others. On Wed, Apr 20, 2016 at 10:05 PM, Paul Angus <paul.an...@shapeblue.com> wrote: > Hi, > > I don't think that it's a question of importance. INFO vs DEBUG should be > telling you different things. ERROR and WARN are also quite different. > In general I'm targeting the cloud operational admins, the people who need > to know the health of their cloud and deal with issues as they're > constantly reading the log. > > However, I'm not proposing to add or remove any messages, just revisit the > categorisation. If the 'operator' has their logging on debug they'd > actually see the same messages. The idea is to make turning the logging > down to INFO feasible. > > You'd turn the logging back up to pick up code issues rather than just > operational issues. > > Some VERY simplified definitions might be: > > DEBUG: inner workings of CloudStack that only a developer can 'understand' > INFO: actions that CloudStack is performing or information analysis (host > 5 is full so not going to use it). > WARN: this isn't good > ERROR: something just didn't work. > > > Grabbing a couple of examples > > these should be debug - I can't 'do' anything with this information. > > INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > (AsyncJobMgr-Heartbeat-1:ctx-4119d5bc) Begin cleanup expired async-jobs > INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > (AsyncJobMgr-Heartbeat-1:ctx-4119d5bc) End cleanup expired async-jobs > INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-2:ctx-3209f65c) Scan > hung worker VM to recycle > INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-113:ctx-56d9f9f2) > Scan hung worker VM to recycle > > Whereas this should be WARN. I wouldn't want to lose this by switching off > DEBUG > > DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-91226be7) > Detected management node left, id:2, nodeIP:10.2.0.6 > DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-0bccd078) > Detected management node left, id:2, nodeIP:10.2.0.6 > > > > > Kind regards, > > Paul Angus > > Regards, > > Paul Angus > > paul.an...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK > @shapeblue > > -Original Message- > From: Chaz PC [mailto:dreeems4e...@hotmail.com] > Sent: 20 April 2016 11:32 > To: users@cloudstack.apache.org > Subject: RE: CloudStack Logging > > Hello, > I have a question what is the criteria that you are going to follow to > decide the log record importance. Who is the user that you will design the > logging process for. Is it for finding issues with the installation or is > it for security breaches or auditing. > These questions are good to put in mind when designing the logging process. > > > Sent from my Samsung Galaxy smartphone. > Original message > From: Paul Angus <paul.an...@shapeblue.com> > Date: 20/04/2016 1:01 PM (GMT+04:00) To: > d...@cloudstack.apache.org, users@cloudstack.apache.org > Subject: RE: CloudStack Logging Hi Simon, > > My gut says that it's probably not worth going back before 4.3. There > have been large changes since then but I think that it's better to get as > much data as possible, limiting to 4.6+ might not give a particularly large > install base. > > > Kind regards, > > Paul Angus > > Regards, > > Paul Angus > > paul.an...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue > > -Original Message- > From: Simon Weller [mailto:swel...@ena.com] > Sent: 19 April 2016 21:51 > To: users@cloudstack.apache.org; d...@cloudstack.apache.org > Subject: Re: CloudStack Logging > > Paul, > > Are you wanting for focus on logs on releases later than a particular > version (e.g. 4.6)? > > - Si > > From: Paul Angus <paul.an...@shapeblue.com> > Sent: Tuesday, April 19, 2016 10:55 AM > To: users@cloudstack.apache.org; d...@cloudstack.apache.org > Subject: RE: CloudStack Logging > > No problem. > > Good question - Any and all logs. Often, it's the mundane stuff which > ob
RE: CloudStack Logging
Hi, I don't think that it's a question of importance. INFO vs DEBUG should be telling you different things. ERROR and WARN are also quite different. In general I'm targeting the cloud operational admins, the people who need to know the health of their cloud and deal with issues as they're constantly reading the log. However, I'm not proposing to add or remove any messages, just revisit the categorisation. If the 'operator' has their logging on debug they'd actually see the same messages. The idea is to make turning the logging down to INFO feasible. You'd turn the logging back up to pick up code issues rather than just operational issues. Some VERY simplified definitions might be: DEBUG: inner workings of CloudStack that only a developer can 'understand' INFO: actions that CloudStack is performing or information analysis (host 5 is full so not going to use it). WARN: this isn't good ERROR: something just didn't work. Grabbing a couple of examples these should be debug - I can't 'do' anything with this information. INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-4119d5bc) Begin cleanup expired async-jobs INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-4119d5bc) End cleanup expired async-jobs INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-2:ctx-3209f65c) Scan hung worker VM to recycle INFO [c.c.h.v.r.VmwareResource] (DirectAgentCronJob-113:ctx-56d9f9f2) Scan hung worker VM to recycle Whereas this should be WARN. I wouldn't want to lose this by switching off DEBUG DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-91226be7) Detected management node left, id:2, nodeIP:10.2.0.6 DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-0bccd078) Detected management node left, id:2, nodeIP:10.2.0.6 Kind regards, Paul Angus Regards, Paul Angus paul.an...@shapeblue.com www.shapeblue.com 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue -Original Message- From: Chaz PC [mailto:dreeems4e...@hotmail.com] Sent: 20 April 2016 11:32 To: users@cloudstack.apache.org Subject: RE: CloudStack Logging Hello, I have a question what is the criteria that you are going to follow to decide the log record importance. Who is the user that you will design the logging process for. Is it for finding issues with the installation or is it for security breaches or auditing. These questions are good to put in mind when designing the logging process. Sent from my Samsung Galaxy smartphone. Original message From: Paul Angus <paul.an...@shapeblue.com> Date: 20/04/2016 1:01 PM (GMT+04:00) To: d...@cloudstack.apache.org, users@cloudstack.apache.org Subject: RE: CloudStack Logging Hi Simon, My gut says that it's probably not worth going back before 4.3. There have been large changes since then but I think that it's better to get as much data as possible, limiting to 4.6+ might not give a particularly large install base. Kind regards, Paul Angus Regards, Paul Angus paul.an...@shapeblue.com www.shapeblue.com 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue -Original Message- From: Simon Weller [mailto:swel...@ena.com] Sent: 19 April 2016 21:51 To: users@cloudstack.apache.org; d...@cloudstack.apache.org Subject: Re: CloudStack Logging Paul, Are you wanting for focus on logs on releases later than a particular version (e.g. 4.6)? - Si From: Paul Angus <paul.an...@shapeblue.com> Sent: Tuesday, April 19, 2016 10:55 AM To: users@cloudstack.apache.org; d...@cloudstack.apache.org Subject: RE: CloudStack Logging No problem. Good question - Any and all logs. Often, it's the mundane stuff which obscures the messages that you really need to see! Kind regards, Paul Angus Regards, Paul Angus paul.an...@shapeblue.com www.shapeblue.com 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue -Original Message- From: williamstev...@gmail.com [mailto:williamstev...@gmail.com] On Behalf Of Will Stevens Sent: 19 April 2016 16:53 To: d...@cloudstack.apache.org Cc: users@cloudstack.apache.org Subject: Re: CloudStack Logging Thanks Paul. Are you looking for only logs with errors or are you just looking for just any management server logs? *Will STEVENS* Lead Developer *CloudOps* *| *Cloud Solutions Experts 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_ On Tue, Apr 19, 2016 at 11:45 AM, Paul Angus <paul.an...@shapeblue.com> wrote: > Thanks Will, > > As far as I know anything sensitive is removed or obfuscated (vcenter > password displayed as h**). IP addresses and hostnames are shown. > Users may want to use a tool like SED to replace IP addresses or > internal domain names > > http://forums.qrz.com/index.php?threads/remove-ip-addresses-from-log-f > iles.204196/ > > > Kind regards, > > Paul Angus > > Regard
RE: CloudStack Logging
Hello, I have a question what is the criteria that you are going to follow to decide the log record importance. Who is the user that you will design the logging process for. Is it for finding issues with the installation or is it for security breaches or auditing. These questions are good to put in mind when designing the logging process. Sent from my Samsung Galaxy smartphone. Original message From: Paul Angus <paul.an...@shapeblue.com> Date: 20/04/2016 1:01 PM (GMT+04:00) To: d...@cloudstack.apache.org, users@cloudstack.apache.org Subject: RE: CloudStack Logging Hi Simon, My gut says that it's probably not worth going back before 4.3. There have been large changes since then but I think that it's better to get as much data as possible, limiting to 4.6+ might not give a particularly large install base. Kind regards, Paul Angus Regards, Paul Angus paul.an...@shapeblue.com www.shapeblue.com 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue -Original Message- From: Simon Weller [mailto:swel...@ena.com] Sent: 19 April 2016 21:51 To: users@cloudstack.apache.org; d...@cloudstack.apache.org Subject: Re: CloudStack Logging Paul, Are you wanting for focus on logs on releases later than a particular version (e.g. 4.6)? - Si From: Paul Angus <paul.an...@shapeblue.com> Sent: Tuesday, April 19, 2016 10:55 AM To: users@cloudstack.apache.org; d...@cloudstack.apache.org Subject: RE: CloudStack Logging No problem. Good question - Any and all logs. Often, it's the mundane stuff which obscures the messages that you really need to see! Kind regards, Paul Angus Regards, Paul Angus paul.an...@shapeblue.com www.shapeblue.com 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue -Original Message- From: williamstev...@gmail.com [mailto:williamstev...@gmail.com] On Behalf Of Will Stevens Sent: 19 April 2016 16:53 To: d...@cloudstack.apache.org Cc: users@cloudstack.apache.org Subject: Re: CloudStack Logging Thanks Paul. Are you looking for only logs with errors or are you just looking for just any management server logs? *Will STEVENS* Lead Developer *CloudOps* *| *Cloud Solutions Experts 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_ On Tue, Apr 19, 2016 at 11:45 AM, Paul Angus <paul.an...@shapeblue.com> wrote: > Thanks Will, > > As far as I know anything sensitive is removed or obfuscated (vcenter > password displayed as h**). IP addresses and hostnames are shown. > Users may want to use a tool like SED to replace IP addresses or > internal domain names > > http://forums.qrz.com/index.php?threads/remove-ip-addresses-from-log-f > iles.204196/ > > > Kind regards, > > Paul Angus > > Regards, > > Paul Angus > > paul.an...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue > > -Original Message- > From: williamstev...@gmail.com [mailto:williamstev...@gmail.com] On > Behalf Of Will Stevens > Sent: 19 April 2016 16:40 > To: d...@cloudstack.apache.org > Cc: users@cloudstack.apache.org > Subject: Re: CloudStack Logging > > I like this initiative Paul. I just wanted to check with you. Do you > know if there is any confidential information like credentials or the > like that is logged in the management server log? I thought we had > done a push at one point to make sure no sensitive data was logged, > but I don't remember for sure. I think knowing the answer to this > will help people feel more comfortable uploading logs. > > Thanks for the valuable initiative... > > *Will STEVENS* > Lead Developer > > *CloudOps* *| *Cloud Solutions Experts > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw > @CloudOps_ > > On Tue, Apr 19, 2016 at 11:34 AM, Paul Angus > <paul.an...@shapeblue.com> > wrote: > > > Hi All, > > > > I'm running an initiative to improve CloudStack logging. I'm going > > to be analysing logs and re-categorising the DEBUG, INFO, WARN and > > ERROR where appropriate such that we can reduce the default logging > > level to INFO without losing important troubleshooting information, > > while at the same time making the management-server log far more > > readable for operational admins. > > > > To achieve this I need logs to work with. > > > > Please could anyone interested upload logs to: > > https://shapeblue.brickftp.com > > > > The logs will be treated in the strictest confidence and public > > download of the logs will not be available. > > I'm especially interested in logs which you found particularly > > difficult to read or specific messages which you think are > > misscategor
RE: CloudStack Logging
Hi Simon, My gut says that it's probably not worth going back before 4.3. There have been large changes since then but I think that it's better to get as much data as possible, limiting to 4.6+ might not give a particularly large install base. Kind regards, Paul Angus Regards, Paul Angus paul.an...@shapeblue.com www.shapeblue.com 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue -Original Message- From: Simon Weller [mailto:swel...@ena.com] Sent: 19 April 2016 21:51 To: users@cloudstack.apache.org; d...@cloudstack.apache.org Subject: Re: CloudStack Logging Paul, Are you wanting for focus on logs on releases later than a particular version (e.g. 4.6)? - Si From: Paul Angus <paul.an...@shapeblue.com> Sent: Tuesday, April 19, 2016 10:55 AM To: users@cloudstack.apache.org; d...@cloudstack.apache.org Subject: RE: CloudStack Logging No problem. Good question - Any and all logs. Often, it's the mundane stuff which obscures the messages that you really need to see! Kind regards, Paul Angus Regards, Paul Angus paul.an...@shapeblue.com www.shapeblue.com 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue -Original Message- From: williamstev...@gmail.com [mailto:williamstev...@gmail.com] On Behalf Of Will Stevens Sent: 19 April 2016 16:53 To: d...@cloudstack.apache.org Cc: users@cloudstack.apache.org Subject: Re: CloudStack Logging Thanks Paul. Are you looking for only logs with errors or are you just looking for just any management server logs? *Will STEVENS* Lead Developer *CloudOps* *| *Cloud Solutions Experts 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_ On Tue, Apr 19, 2016 at 11:45 AM, Paul Angus <paul.an...@shapeblue.com> wrote: > Thanks Will, > > As far as I know anything sensitive is removed or obfuscated (vcenter > password displayed as h**). IP addresses and hostnames are shown. > Users may want to use a tool like SED to replace IP addresses or > internal domain names > > http://forums.qrz.com/index.php?threads/remove-ip-addresses-from-log-f > iles.204196/ > > > Kind regards, > > Paul Angus > > Regards, > > Paul Angus > > paul.an...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue > > -Original Message- > From: williamstev...@gmail.com [mailto:williamstev...@gmail.com] On > Behalf Of Will Stevens > Sent: 19 April 2016 16:40 > To: d...@cloudstack.apache.org > Cc: users@cloudstack.apache.org > Subject: Re: CloudStack Logging > > I like this initiative Paul. I just wanted to check with you. Do you > know if there is any confidential information like credentials or the > like that is logged in the management server log? I thought we had > done a push at one point to make sure no sensitive data was logged, > but I don't remember for sure. I think knowing the answer to this > will help people feel more comfortable uploading logs. > > Thanks for the valuable initiative... > > *Will STEVENS* > Lead Developer > > *CloudOps* *| *Cloud Solutions Experts > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw > @CloudOps_ > > On Tue, Apr 19, 2016 at 11:34 AM, Paul Angus > <paul.an...@shapeblue.com> > wrote: > > > Hi All, > > > > I'm running an initiative to improve CloudStack logging. I'm going > > to be analysing logs and re-categorising the DEBUG, INFO, WARN and > > ERROR where appropriate such that we can reduce the default logging > > level to INFO without losing important troubleshooting information, > > while at the same time making the management-server log far more > > readable for operational admins. > > > > To achieve this I need logs to work with. > > > > Please could anyone interested upload logs to: > > https://shapeblue.brickftp.com > > > > The logs will be treated in the strictest confidence and public > > download of the logs will not be available. > > I'm especially interested in logs which you found particularly > > difficult to read or specific messages which you think are > > misscategorised > > > > If you wish, put comments at the top of the logs or send a message > > directly or via the mailing lists pointing out specific issues. > > Please compress the logs! > > > > Kind regards, > > > > Paul Angus > > > > > > Regards, > > > > Paul Angus > > > > paul.an...@shapeblue.com > > www.shapeblue.com > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue > > >
Re: CloudStack Logging
Paul, Are you wanting for focus on logs on releases later than a particular version (e.g. 4.6)? - Si From: Paul Angus <paul.an...@shapeblue.com> Sent: Tuesday, April 19, 2016 10:55 AM To: users@cloudstack.apache.org; d...@cloudstack.apache.org Subject: RE: CloudStack Logging No problem. Good question - Any and all logs. Often, it's the mundane stuff which obscures the messages that you really need to see! Kind regards, Paul Angus Regards, Paul Angus paul.an...@shapeblue.com www.shapeblue.com 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue -Original Message- From: williamstev...@gmail.com [mailto:williamstev...@gmail.com] On Behalf Of Will Stevens Sent: 19 April 2016 16:53 To: d...@cloudstack.apache.org Cc: users@cloudstack.apache.org Subject: Re: CloudStack Logging Thanks Paul. Are you looking for only logs with errors or are you just looking for just any management server logs? *Will STEVENS* Lead Developer *CloudOps* *| *Cloud Solutions Experts 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_ On Tue, Apr 19, 2016 at 11:45 AM, Paul Angus <paul.an...@shapeblue.com> wrote: > Thanks Will, > > As far as I know anything sensitive is removed or obfuscated (vcenter > password displayed as h**). IP addresses and hostnames are shown. > Users may want to use a tool like SED to replace IP addresses or > internal domain names > > http://forums.qrz.com/index.php?threads/remove-ip-addresses-from-log-f > iles.204196/ > > > Kind regards, > > Paul Angus > > Regards, > > Paul Angus > > paul.an...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue > > -Original Message- > From: williamstev...@gmail.com [mailto:williamstev...@gmail.com] On > Behalf Of Will Stevens > Sent: 19 April 2016 16:40 > To: d...@cloudstack.apache.org > Cc: users@cloudstack.apache.org > Subject: Re: CloudStack Logging > > I like this initiative Paul. I just wanted to check with you. Do you > know if there is any confidential information like credentials or the > like that is logged in the management server log? I thought we had > done a push at one point to make sure no sensitive data was logged, > but I don't remember for sure. I think knowing the answer to this > will help people feel more comfortable uploading logs. > > Thanks for the valuable initiative... > > *Will STEVENS* > Lead Developer > > *CloudOps* *| *Cloud Solutions Experts > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw > @CloudOps_ > > On Tue, Apr 19, 2016 at 11:34 AM, Paul Angus > <paul.an...@shapeblue.com> > wrote: > > > Hi All, > > > > I'm running an initiative to improve CloudStack logging. I'm going > > to be analysing logs and re-categorising the DEBUG, INFO, WARN and > > ERROR where appropriate such that we can reduce the default logging > > level to INFO without losing important troubleshooting information, > > while at the same time making the management-server log far more > > readable for operational admins. > > > > To achieve this I need logs to work with. > > > > Please could anyone interested upload logs to: > > https://shapeblue.brickftp.com > > > > The logs will be treated in the strictest confidence and public > > download of the logs will not be available. > > I'm especially interested in logs which you found particularly > > difficult to read or specific messages which you think are > > misscategorised > > > > If you wish, put comments at the top of the logs or send a message > > directly or via the mailing lists pointing out specific issues. > > Please compress the logs! > > > > Kind regards, > > > > Paul Angus > > > > > > Regards, > > > > Paul Angus > > > > paul.an...@shapeblue.com > > www.shapeblue.com > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue > > >
RE: CloudStack Logging
No problem. Good question - Any and all logs. Often, it's the mundane stuff which obscures the messages that you really need to see! Kind regards, Paul Angus Regards, Paul Angus paul.an...@shapeblue.com www.shapeblue.com 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue -Original Message- From: williamstev...@gmail.com [mailto:williamstev...@gmail.com] On Behalf Of Will Stevens Sent: 19 April 2016 16:53 To: d...@cloudstack.apache.org Cc: users@cloudstack.apache.org Subject: Re: CloudStack Logging Thanks Paul. Are you looking for only logs with errors or are you just looking for just any management server logs? *Will STEVENS* Lead Developer *CloudOps* *| *Cloud Solutions Experts 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_ On Tue, Apr 19, 2016 at 11:45 AM, Paul Angus <paul.an...@shapeblue.com> wrote: > Thanks Will, > > As far as I know anything sensitive is removed or obfuscated (vcenter > password displayed as h**). IP addresses and hostnames are shown. > Users may want to use a tool like SED to replace IP addresses or > internal domain names > > http://forums.qrz.com/index.php?threads/remove-ip-addresses-from-log-f > iles.204196/ > > > Kind regards, > > Paul Angus > > Regards, > > Paul Angus > > paul.an...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue > > -Original Message- > From: williamstev...@gmail.com [mailto:williamstev...@gmail.com] On > Behalf Of Will Stevens > Sent: 19 April 2016 16:40 > To: d...@cloudstack.apache.org > Cc: users@cloudstack.apache.org > Subject: Re: CloudStack Logging > > I like this initiative Paul. I just wanted to check with you. Do you > know if there is any confidential information like credentials or the > like that is logged in the management server log? I thought we had > done a push at one point to make sure no sensitive data was logged, > but I don't remember for sure. I think knowing the answer to this > will help people feel more comfortable uploading logs. > > Thanks for the valuable initiative... > > *Will STEVENS* > Lead Developer > > *CloudOps* *| *Cloud Solutions Experts > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw > @CloudOps_ > > On Tue, Apr 19, 2016 at 11:34 AM, Paul Angus > <paul.an...@shapeblue.com> > wrote: > > > Hi All, > > > > I'm running an initiative to improve CloudStack logging. I'm going > > to be analysing logs and re-categorising the DEBUG, INFO, WARN and > > ERROR where appropriate such that we can reduce the default logging > > level to INFO without losing important troubleshooting information, > > while at the same time making the management-server log far more > > readable for operational admins. > > > > To achieve this I need logs to work with. > > > > Please could anyone interested upload logs to: > > https://shapeblue.brickftp.com > > > > The logs will be treated in the strictest confidence and public > > download of the logs will not be available. > > I'm especially interested in logs which you found particularly > > difficult to read or specific messages which you think are > > misscategorised > > > > If you wish, put comments at the top of the logs or send a message > > directly or via the mailing lists pointing out specific issues. > > Please compress the logs! > > > > Kind regards, > > > > Paul Angus > > > > > > Regards, > > > > Paul Angus > > > > paul.an...@shapeblue.com > > www.shapeblue.com > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue > > >
Re: CloudStack Logging
Thanks Paul. Are you looking for only logs with errors or are you just looking for just any management server logs? *Will STEVENS* Lead Developer *CloudOps* *| *Cloud Solutions Experts 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_ On Tue, Apr 19, 2016 at 11:45 AM, Paul Angus <paul.an...@shapeblue.com> wrote: > Thanks Will, > > As far as I know anything sensitive is removed or obfuscated (vcenter > password displayed as h**). IP addresses and hostnames are shown. > Users may want to use a tool like SED to replace IP addresses or internal > domain names > > http://forums.qrz.com/index.php?threads/remove-ip-addresses-from-log-files.204196/ > > > Kind regards, > > Paul Angus > > Regards, > > Paul Angus > > paul.an...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK > @shapeblue > > -Original Message- > From: williamstev...@gmail.com [mailto:williamstev...@gmail.com] On > Behalf Of Will Stevens > Sent: 19 April 2016 16:40 > To: d...@cloudstack.apache.org > Cc: users@cloudstack.apache.org > Subject: Re: CloudStack Logging > > I like this initiative Paul. I just wanted to check with you. Do you > know if there is any confidential information like credentials or the like > that is logged in the management server log? I thought we had done a push > at one point to make sure no sensitive data was logged, but I don't > remember for sure. I think knowing the answer to this will help people > feel more comfortable uploading logs. > > Thanks for the valuable initiative... > > *Will STEVENS* > Lead Developer > > *CloudOps* *| *Cloud Solutions Experts > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw > @CloudOps_ > > On Tue, Apr 19, 2016 at 11:34 AM, Paul Angus <paul.an...@shapeblue.com> > wrote: > > > Hi All, > > > > I'm running an initiative to improve CloudStack logging. I'm going to > > be analysing logs and re-categorising the DEBUG, INFO, WARN and ERROR > > where appropriate such that we can reduce the default logging level to > > INFO without losing important troubleshooting information, while at > > the same time making the management-server log far more readable for > > operational admins. > > > > To achieve this I need logs to work with. > > > > Please could anyone interested upload logs to: > > https://shapeblue.brickftp.com > > > > The logs will be treated in the strictest confidence and public > > download of the logs will not be available. > > I'm especially interested in logs which you found particularly > > difficult to read or specific messages which you think are > > misscategorised > > > > If you wish, put comments at the top of the logs or send a message > > directly or via the mailing lists pointing out specific issues. > > Please compress the logs! > > > > Kind regards, > > > > Paul Angus > > > > > > Regards, > > > > Paul Angus > > > > paul.an...@shapeblue.com > > www.shapeblue.com > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue > > >
RE: CloudStack Logging
Thanks Will, As far as I know anything sensitive is removed or obfuscated (vcenter password displayed as h**). IP addresses and hostnames are shown. Users may want to use a tool like SED to replace IP addresses or internal domain names http://forums.qrz.com/index.php?threads/remove-ip-addresses-from-log-files.204196/ Kind regards, Paul Angus Regards, Paul Angus paul.an...@shapeblue.com www.shapeblue.com 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue -Original Message- From: williamstev...@gmail.com [mailto:williamstev...@gmail.com] On Behalf Of Will Stevens Sent: 19 April 2016 16:40 To: d...@cloudstack.apache.org Cc: users@cloudstack.apache.org Subject: Re: CloudStack Logging I like this initiative Paul. I just wanted to check with you. Do you know if there is any confidential information like credentials or the like that is logged in the management server log? I thought we had done a push at one point to make sure no sensitive data was logged, but I don't remember for sure. I think knowing the answer to this will help people feel more comfortable uploading logs. Thanks for the valuable initiative... *Will STEVENS* Lead Developer *CloudOps* *| *Cloud Solutions Experts 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_ On Tue, Apr 19, 2016 at 11:34 AM, Paul Angus <paul.an...@shapeblue.com> wrote: > Hi All, > > I'm running an initiative to improve CloudStack logging. I'm going to > be analysing logs and re-categorising the DEBUG, INFO, WARN and ERROR > where appropriate such that we can reduce the default logging level to > INFO without losing important troubleshooting information, while at > the same time making the management-server log far more readable for > operational admins. > > To achieve this I need logs to work with. > > Please could anyone interested upload logs to: > https://shapeblue.brickftp.com > > The logs will be treated in the strictest confidence and public > download of the logs will not be available. > I'm especially interested in logs which you found particularly > difficult to read or specific messages which you think are > misscategorised > > If you wish, put comments at the top of the logs or send a message > directly or via the mailing lists pointing out specific issues. > Please compress the logs! > > Kind regards, > > Paul Angus > > > Regards, > > Paul Angus > > paul.an...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue >
Re: CloudStack Logging
I like this initiative Paul. I just wanted to check with you. Do you know if there is any confidential information like credentials or the like that is logged in the management server log? I thought we had done a push at one point to make sure no sensitive data was logged, but I don't remember for sure. I think knowing the answer to this will help people feel more comfortable uploading logs. Thanks for the valuable initiative... *Will STEVENS* Lead Developer *CloudOps* *| *Cloud Solutions Experts 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw @CloudOps_ On Tue, Apr 19, 2016 at 11:34 AM, Paul Anguswrote: > Hi All, > > I'm running an initiative to improve CloudStack logging. I'm going to be > analysing logs and re-categorising the DEBUG, INFO, WARN and ERROR where > appropriate such that we can reduce the default logging level to INFO > without losing important troubleshooting information, while at the same > time making the management-server log far more readable for operational > admins. > > To achieve this I need logs to work with. > > Please could anyone interested upload logs to: > https://shapeblue.brickftp.com > > The logs will be treated in the strictest confidence and public download > of the logs will not be available. > I'm especially interested in logs which you found particularly difficult > to read or specific messages which you think are misscategorised > > If you wish, put comments at the top of the logs or send a message > directly or via the mailing lists pointing out specific issues. > Please compress the logs! > > Kind regards, > > Paul Angus > > > Regards, > > Paul Angus > > paul.an...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK > @shapeblue >