Re: ~charmers Application - David Britton
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Required +1 count met (and exceeded!). Welcome to the team! - -- James Page Ubuntu and Debian Developer james.p...@ubuntu.com jamesp...@debian.org -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQIcBAEBCAAGBQJT7coaAAoJEL/srsug59jDpuYP/j/eKSMMuuv+gBLUzZ49/lkX 18GT+ntt2pqZv2+C4MVYdc+cuVfabsRHQFSeu7gcyFFBjayjiz6hVjeyuKewwjwE Bao3J1BCJL4geaRcFONltODHbYwnZjVHDmzgwW3MB8VHQj2f8ilPhQL4Y1sd/OrY j4ASB4ljRSZ+iXnb+9NmL2Q7LU2ByCDc1Wc6OidOjZuE1qtumLdoFGiAaEnrK1Ju gH6tCcjolOCpTvVcZMdR7bYujlxYUCHVd4mccR6/edMu0g6mOwIB9+rGVbCgjo35 djgCMeGUQO7KF+0lvsbmrBad1Wh3reUM1EKhi0vv+apGpmeRI7RBimCJyBREp4FC eOhDzkk2VOgOL8kI3iz0d07IDiMR9DobL2x4Q3HnO17/Qe6P2oeNYrt/T+C0ONBn hPI/gr5hNyka15kVU4IjF4RppoxzQIzwznF2due1LlhSB1uUhwl+LlPzRFedxX0I rIWMbmJccbT4R3rDExc9i0ty0NZ/tUJNWvfLgnIkKAdr+esXAcIWS4PYpsBHEwVm rPiwe65Ov1pZOPIRIfqSKgGYXsR5OHCAA0aWxC7PCLyJr6agybLMLAuHwbn/02AD hpHNS3MvbApHlBYURvRTZnryDDU1jo/WfJ2UWYFlIHi4W5lZjdrR79GwyVi75ttS 7LqeRNGg5kXGJtUe/HAc =qWOW -END PGP SIGNATURE- -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: getting rid of all-machines.log
On Fri, Aug 15, 2014 at 7:36 AM, Gabriel Samfira gsamf...@cloudbasesolutions.com wrote: I think this thread has become a bit lengthy, and we have started to loose perspective on what we are actually trying to accomplish. agreed. afaics that's how do we support logging on windows. Gustavo's idea to save the logs to is awesome and it works across platforms, allows immense flexibility and would give people a powerful tool. We should deffinately aspire to get that done sooner rather then later. However, at this point in time its only an idea, without a clear blueprint. agreed. What Nate is proposing *already exists*, its tangible, proposed as a PR and improves the way juju handles logs. you'll have to be more specific, there's been a shotgun of statements in this thread, touching on logstash, aggregation removal, rsyslog removal, log rotation, deferring to stderr/stdout, 12factor apps, working with ha state servers, etc. The only thing I see missing, that might ease people's minds is a --log-file option (that works with --debug) to actually enforce the usage of a log file. If we omit that option, then juju should just log to stdout/stderr. So we get to keep what we have, but also solve a huge PITA on Windows or any other platform that have limitations in this respect, with a minimal change... afaics your referencing your branch ( https://github.com/gabriel-samfira/syslog) which will directly send logs to a remote aggregating syslog server on windows nodes. with regard to the default service behavior on windows, where does stdout/stderr go for a service? isn't the expected behavior for a windows service to use the event log facility? ie. to be inline with expected windows behavior and extant juju semantics, shouldn't we have multiple handlers on the log facility, one to rsyslog, and one to event log on windows. Please keep in mind that its better to move forward no matter how small the steps, then to just stand still while we figure out the perfect logging system. I would much rather have windows support today, then 2 months from now when someone actually gets around to implement a *new* logging system. This should not be a discussion about which logging system is _best_. This should be a discussion about which logging system is _better_ and available *now*. Otherwise we risk of getting caught up in details and loose sight of our actual goal. Just my 2 cents. Regards, Gabriel Thanks, Kapil On 14.08.2014 23:47, Kapil Thangavelu wrote: On Thu, Aug 14, 2014 at 2:14 PM, Nate Finch nate.fi...@canonical.com wrote: I didn't bring up 12 factor, it's irrelevant to my argument. I'm trying to make our product simpler and easier to maintain. That is all. If there's another cross-platform solution that we can use, I'd be happy to consider it. We have to change the code to support Windows. I'd rather the diff be +50 -150 than +75 -0. I don't know how to state it any simpler than that. the abrogation of responsibility which is what ic you adocating for in this thread, also makes our product quite a lot less usable imo... Our product is a distributed system with emergent behavior. Having a debug log is one of the most useful things you can have to observe the system and back in py days was one of the most used features and it was just a simple dump to the db with querying. Its unfortunate that ability to use it usefully didn't land to core till recently and did so in broken fashion (still requiring internal tag names for filtering).. or lots more people would be using it. Gustavo's suggestion of storing the structured log data in mongo sounds really good to me. Yes, features are work and require code but that sort of implementation is also cross platform portable. The current implementation and proposed alternatives I find somewhat ridicolous in that we basically dump structured data into an unstructured format only to reparse it every time we look at it (or ingest into logstash) given that we already have the structured data. Asking people to setup one of those distributed log aggregation systems systems and configure them is a huge task, and anyone suggesting punting that to an end user or charm developer has never setup one up themselves i suspect. ie. an analogy imo http://xahlee.info/comp/i/fault-tolerance_NoSQL.png As for the operations follks who do have them.. we can continue sending messages send to local syslog and let them collect per their preference. -k On Thu, Aug 14, 2014 at 1:35 PM, Gustavo Niemeyer gustavo.nieme...@canonical.com wrote: On Thu, Aug 14, 2014 at 1:35 PM, Nate Finch nate.fi...@canonical.com wrote: On Thu, Aug 14, 2014 at 12:24 PM, Gustavo Niemeyer gustavo.nieme...@canonical.com wrote: Why support two things when you can support just one? Just to be clear, you really mean why support two existing and well known things when I can implement a third thing, right?
Re: getting rid of all-machines.log
On 15.08.2014 16:35, Kapil Thangavelu wrote: On Fri, Aug 15, 2014 at 7:36 AM, Gabriel Samfira gsamf...@cloudbasesolutions.commailto:gsamf...@cloudbasesolutions.com wrote: I think this thread has become a bit lengthy, and we have started to loose perspective on what we are actually trying to accomplish. agreed. afaics that's how do we support logging on windows. Gustavo's idea to save the logs to is awesome and it works across platforms, allows immense flexibility and would give people a powerful tool. We should deffinately aspire to get that done sooner rather then later. However, at this point in time its only an idea, without a clear blueprint. agreed. What Nate is proposing *already exists*, its tangible, proposed as a PR and improves the way juju handles logs. you'll have to be more specific, there's been a shotgun of statements in this thread, touching on logstash, aggregation removal, rsyslog removal, log rotation, deferring to stderr/stdout, 12factor apps, working with ha state servers, etc. I was referring to Nate's lumberjack package (PR seams to be gone) and the syslog streaming change. Lumberjack works on windows as well. While the rest have been mentioned, they are just ideas that may or may not be implemented in the future. I don't think Nate started this thread with the intent of prioritising this *now*. From what I can tell it was just a thought that was meant to start a discussion to implement something better in the *future*. So far Gustavo's idea seams like the best choice with the most benefits in the long term. I would take that from this whole thread, and mark the case of log aggregation *closed* (of course pending a blueprint). The only thing I see missing, that might ease people's minds is a --log-file option (that works with --debug) to actually enforce the usage of a log file. If we omit that option, then juju should just log to stdout/stderr. So we get to keep what we have, but also solve a huge PITA on Windows or any other platform that have limitations in this respect, with a minimal change... afaics your referencing your branch (https://github.com/gabriel-samfira/syslog) which will directly send logs to a remote aggregating syslog server on windows nodes. with regard to the default service behavior on windows, where does stdout/stderr go for a service? isn't the expected behavior for a windows service to use the event log facility? ie. to be inline with expected windows behavior and extant juju semantics, shouldn't we have multiple handlers on the log facility, one to rsyslog, and one to event log on windows. While the initial scope of this thread was to see if we really do need to save an all-machines.log via syslog (and apparently the CI needs it) instead of using a better tool for this particular job (like logstash...not not necessarily that), it has gone off into a bunch of tangents, one of which is using lumberjack to write the logs and rotate them instead of having upstart write them via redirect, and then logrotate rotate and archive them(2 external and platform specific dependencies as opposed to one cross platform implementation). Windows services have no way of capturing stdout/stderr in the same way upstart does (solved by using Nate's lumberjack). It is simply lost, or the service will fail because it cannot write to stdout/stderr (there is no console present). The syslog change you mentioned is just half of the solution, and will allow the windows version to log remotely and be in-line with the ubuntu version without needing to install 3rd party applications. That was my initial goal. The fact that it removes rsyslog as a dependency from non-bootstrap nodes on Ubuntu as well, is in my opinion a bonus :). You get to remove an external dependency which you no longer have to configure separately, and we finally do ssl certificate checking of the rsyslog server on the state machine (right now the way the forwarders are configured, will ignore the SSL errors). As for logging to event log, that is certainly a good option, and I will look into it. If it solves the local logging issue, I am happy with it and will soon propose a PR. If there is a package to do this that implements an io.Writer, it should be trivial. So to summarize: * I will be looking into event log for local windows logging. This will probably require writing a package. * the syslog change will solve in the sort term, the aggregation issue from Windows nodes (when something better comes along, I will personally send a case of beer or ice-cream...or both, to whomever removes syslog as a dependency) * lumberjack works *now* for local logging on both Windows and Ubuntu. It simply removes 2 dependencies (for logging) with just a few lines of code... As I have mentioned, I am willing to adapt to any decision you guys make (I am just an outsider), as long as that decision does not imply 2 more months and 3 extra packages just to get a
First customer pain point pull request - default-hook
Just wanted to let people know that Moonstone is ramping up on the customer pain points, even ahead of the full spec and prioritization. I had talked to Jorge and Marco about what they thought was important, and they pointed out a couple of low hanging fruit. This was one of them. Many charms these days only contain one real hook script, and the rest are all just symlinks to the real one. (because no one wants to write 20 scripts) This is kind of a pain in the ass for charm writers, and doesn't work well on Windows (Windows symlink support is terrible). So, why not just have a default hook that gets called if the real hook isn't there? That's what I implemented today: https://github.com/juju/juju/pull/528 There's new hook in town: default-hook. If it exists and a hook gets called that doesn't have a corresponding hook file, default-hook gets called with the name of the original hook as its first argument (arg[1]). That's it. If/when this PR is accepted, Marco is planning to update charmhelpers to make it automatically recognize when the default-hook is called, and get the hook name from arg[1] instead of arg[0], so current scripts wouldn't even need to change - they'd just need the new charmhelpers, rename the one true script to default-hook, and delete all their symlinks. Bam. Moonstone is very excited to be working to make Juju easier for charm developers, and we'll see more improvements coming next week. -Nate -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: First customer pain point pull request - default-hook
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 14-08-15 04:36 PM, Nate Finch wrote: There's new hook in town: default-hook. If it exists and a hook gets called that doesn't have a corresponding hook file, default-hook gets called with the name of the original hook as its first argument (arg[1]). That's it. Nice! Thank you. Aaron -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQEcBAEBAgAGBQJT7nVvAAoJEK84cMOcf+9h90UH/RMVabfJp4Ynkueh5XQiS6mD TPWwY0FVHfpAWEIbnQTQpnmkhMzSOKIFy0fkkXkEx4jSUt6I+iNYXdu8T77mA38G 7IZ7HAi+dAzRCrGTIZHsextrs5VpxhdzFJYOxL+TN5VUWYt+U+awSPFn0MlUZfAC 5aUuV3p3KjlHByLNT7ob3eMzR2mwylP+AS/9UgiojbUOahlff/9y83dYqkCDYzih C2rlwf0Wal12svu70ifggGKWcnF/eiwSm4TQjJsfMdCfw0gSg4ICgmIbWQ78OytJ AM4UBk1/Ue94dUm3YP+lcgAqJCC9GW5HksCFN74Qr+4xcnuqYoCJJxpU5fBOTls= =5YwW -END PGP SIGNATURE- -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
