Re: Implementing std.log
The implementation and documentation for std.log is ready for viewing. You can take a look at the doc at http://jsancio.github.com/phobos/phobos/std_log.html. The source code is at https://github.com/jsancio/phobos/blob/master/std/log.d. I had to make some changes to druntime to get the thread id for printing. The module will work without the changes but you wont to see thread ids in your log messages. You can apply the attached patch to your druntime if you want to see thread id in the log. Some comments below. Let me know if you have any comments or suggestions! Thanks, -Jose On Wed, May 18, 2011 at 1:09 PM, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 5/18/11 8:00 AM, Jose Armando Garcia wrote: I also think that having competing logging module is a good thing. This process will result in a better module for phobos and as a result a better module for the user. Clearly there are advantages to competing proposals, but I have mixed feelings about the whole thing, with the negative probably being stronger than the positive. At the end of the day one of us will have their work go to waste. Generally nobody wants their work to go to waste, and in particular I think I'd harm the community's interests by working on redundant stuff when so many other things are in need of attention. The reason I started std.log was out of desperation that everybody wanted to talk about it instead of working on it, and because I figured nobody would implement it the way I thought it needs to be done anyway. That's why I wrote on Apr 21st: Again, I'd _much_ rather prefer if someone just implemented this: http://google-glog.googlecode.com/svn/trunk/doc/glog.html It's simple, to the point, and brings the bacon home. In fact I'm putting dibs on this. I'll implement the thing and make a proposal. That message was meant to prevent exactly what's happening now. Honestly it hurts me that of all things possible, a new and talented contributor (two, to count Jacob's effort on his prototype) chose to work squarely on this one artifact. Right now the two APIs are converging and start differing in minor details, which makes it painfully obvious that at the end of the day two people are working in separation on virtually identical code. I can't afford this. I am ceasing work on std.log (feel free to copy from my code) and I encourage you to work towards a formal proposal. In doing that, I'll be upfront in saying that I'll very strongly advocate staying close to the client interface of std.log as it is now. In particular, every departure from glog (to which both designs now owe a lot) for equivalent functionality is gratuitous unless thoroughly justified. In detail: * initializeLogging(SharedLogger.getCreator(args[0])); adds too much cognitive load, literally at line one. Now I need to know what a shared logger is and what a creator is. Just pass the entire command line and let the log subsystem pick its parameters. Don't use a delegate unless you can't do without. If client code wants to do configuration work, give them functions, don't ask them for a callback. The callback complicates things for no reason. Done. * The name initializeLogging is a gratuitous departure from glog. Use initLogging. Done. * Keep the glog flags as they are. In all likelihood any user of glog would want to define such flags, so we may as well spare them the work. Done. * Pick the logging directory like glog does. Done. * Add programmatic setting of flag names and values before calling initLogging. I think that's a good idea that doesn't depart from glog's design (google cmdline parser uses global names a la FLAGS_xxx). That would allow users to change a flag's name or disable it entirely (by assigning null to them). Assigning to a flag's value prior to calling initLogging would change its default. Assigning to a flag's value after initLogging forces the flag. To simply prevent log to get to any flags, client can call initLogging(args[0 .. 1]). Done. Given how std.log's command line parsing currently works the user can change an option's default but that easily changeable if we really want this. * The obvious action to do with a log is to write to it. I don't want to write: log!info.write(stuff); I want to write: logInfo(stuff); or, fine, log!info(stuff); Done. * The names LOGGING_FATAL_DISABLED etc. are gratuitous departures from glog. Do what glog does adapted to D's naming convention, hence strip_log_xxx. Done. * I don't want to write log!info(args.length 1).write(Arguments: , args[1 .. $]); Putting the condition there makes no sense without the manual. I want to write logInfo.when(args.length 1)(Arguments: , args[1 .. $]); or log!info.when(args.length 1)(Arguments: , args[1 .. $]); which omits the obvious write but adds the non-obvious when. Done. * Factoring out every, first, etc. is an interesting idea but I found no use for it
Re: Implementing std.log
Clever use of __FILE__ and __LINE__ to create unique instantiations of the every() template. :)
Re: Implementing std.log
Am 28.05.2011 20:08, schrieb Andrej Mitrovic: Clever use of __FILE__ and __LINE__ to create unique instantiations of the every() template. :) does this belong in this thread? furthermore __FILE__ and __LINE__ isn't enough for every(...) ; every(...);
Re: Implementing std.log
On 5/28/11, Daniel Gibson metalcae...@gmail.com wrote: Am 28.05.2011 20:08, schrieb Andrej Mitrovic: Clever use of __FILE__ and __LINE__ to create unique instantiations of the every() template. :) does this belong in this thread? I was just commenting the implementation.
Re: Implementing std.log
Am 28.05.2011 20:19, schrieb Andrej Mitrovic: On 5/28/11, Daniel Gibsonmetalcae...@gmail.com wrote: Am 28.05.2011 20:08, schrieb Andrej Mitrovic: Clever use of __FILE__ and __LINE__ to create unique instantiations of the every() template. :) does this belong in this thread? I was just commenting the implementation. Ah ok. I just remembered every from another thread (where using __LINE__ and __FILE__ was discussed) :)
Re: Implementing std.log
On 2011-05-28 10:30, Jose Armando Garcia wrote: The implementation and documentation for std.log is ready for viewing. You can take a look at the doc at http://jsancio.github.com/phobos/phobos/std_log.html. The source code is at https://github.com/jsancio/phobos/blob/master/std/log.d. I had to make some changes to druntime to get the thread id for printing. The module will work without the changes but you wont to see thread ids in your log messages. You can apply the attached patch to your druntime if you want to see thread id in the log. Some comments below. Let me know if you have any comments or suggestions! Thanks, -Jose I'd suggest starting a new thread on it or it's likely that a lot of people will miss this. - Jonathan M Davis
Re: Implementing std.log
Jose Armando Garcia wrote: Just wanted to let everyone know that I am working on having a review wordy std.log. I am almost done with the implementation. I am currently working on improving the documentation. Is there a link that describes the review process. I would like to get std.log as close to final to minimize the back and forth. Also, I will reply to the rest of Andrei's email and Jens's email when we have something to look at (the code and doc in github). Thanks, -Jose I think the review process is meant to be similar to the boost review process, Andrei posted this link a few times: http://www.boost.org/community/reviews.html -- Johannes Pfau
Re: Implementing std.log
Just wanted to let everyone know that I am working on having a review wordy std.log. I am almost done with the implementation. I am currently working on improving the documentation. Is there a link that describes the review process. I would like to get std.log as close to final to minimize the back and forth. Also, I will reply to the rest of Andrei's email and Jens's email when we have something to look at (the code and doc in github). Thanks, -Jose On Wed, May 18, 2011 at 1:09 PM, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 5/18/11 8:00 AM, Jose Armando Garcia wrote: I also think that having competing logging module is a good thing. This process will result in a better module for phobos and as a result a better module for the user. Clearly there are advantages to competing proposals, but I have mixed feelings about the whole thing, with the negative probably being stronger than the positive. At the end of the day one of us will have their work go to waste. Generally nobody wants their work to go to waste, and in particular I think I'd harm the community's interests by working on redundant stuff when so many other things are in need of attention. The reason I started std.log was out of desperation that everybody wanted to talk about it instead of working on it, and because I figured nobody would implement it the way I thought it needs to be done anyway. That's why I wrote on Apr 21st: Again, I'd _much_ rather prefer if someone just implemented this: http://google-glog.googlecode.com/svn/trunk/doc/glog.html It's simple, to the point, and brings the bacon home. In fact I'm putting dibs on this. I'll implement the thing and make a proposal. That message was meant to prevent exactly what's happening now. Honestly it hurts me that of all things possible, a new and talented contributor (two, to count Jacob's effort on his prototype) chose to work squarely on this one artifact. Right now the two APIs are converging and start differing in minor details, which makes it painfully obvious that at the end of the day two people are working in separation on virtually identical code. I can't afford this. I am ceasing work on std.log (feel free to copy from my code) and I encourage you to work towards a formal proposal. In doing that, I'll be upfront in saying that I'll very strongly advocate staying close to the client interface of std.log as it is now. In particular, every departure from glog (to which both designs now owe a lot) for equivalent functionality is gratuitous unless thoroughly justified. In detail: * initializeLogging(SharedLogger.getCreator(args[0])); adds too much cognitive load, literally at line one. Now I need to know what a shared logger is and what a creator is. Just pass the entire command line and let the log subsystem pick its parameters. Don't use a delegate unless you can't do without. If client code wants to do configuration work, give them functions, don't ask them for a callback. The callback complicates things for no reason. * The name initializeLogging is a gratuitous departure from glog. Use initLogging. * Keep the glog flags as they are. In all likelihood any user of glog would want to define such flags, so we may as well spare them the work. * Pick the logging directory like glog does. * Add programmatic setting of flag names and values before calling initLogging. I think that's a good idea that doesn't depart from glog's design (google cmdline parser uses global names a la FLAGS_xxx). That would allow users to change a flag's name or disable it entirely (by assigning null to them). Assigning to a flag's value prior to calling initLogging would change its default. Assigning to a flag's value after initLogging forces the flag. To simply prevent log to get to any flags, client can call initLogging(args[0 .. 1]). * The obvious action to do with a log is to write to it. I don't want to write: log!info.write(stuff); I want to write: logInfo(stuff); or, fine, log!info(stuff); * The names LOGGING_FATAL_DISABLED etc. are gratuitous departures from glog. Do what glog does adapted to D's naming convention, hence strip_log_xxx. * I don't want to write log!info(args.length 1).write(Arguments: , args[1 .. $]); Putting the condition there makes no sense without the manual. I want to write logInfo.when(args.length 1)(Arguments: , args[1 .. $]); or log!info.when(args.length 1)(Arguments: , args[1 .. $]); which omits the obvious write but adds the non-obvious when. * Factoring out every, first, etc. is an interesting idea but I found no use for it outside logging. (That doesn't mean there isn't any, it just means we should think of it because cool things may happen.) That shouldn't harm except when combined with the point above we're forced to: logInfo.when(every(1000))(text); which is self-explanatory but rather verbose. In all honesty log!info(every(9)).write(Every nine);
Re: Implementing std.log
On Sun, 15 May 2011 17:15:38 +0200, Jacob Carlborg wrote: On 2011-05-14 19:04, Andrei Alexandrescu wrote: On 5/9/11 1:52 AM, Andrei Alexandrescu wrote: [snip] I updated my std.log draft. Added a lot of features including formatted writing, delayed logging, and a variety of configuration options. Replaced the redundant log.xyz with logXyz. The implementation is getting close to reviewable form. Documentation: http://d-programming-language.org/phobos-prerelease/std_log.html Source: https://github.com/andralex/phobos Feedback welcome. Thanks, Andrei Why does the user have to manually initialize the library? Why not use a static constructor or lazy initialization? Actually I don't like it at all, that the logging library is configured via command line options. Seems very odd to me in the first place. That is something that should be handled by the application that uses std.log not the library itself. The library should be configurable via regular methods, like log.verbose = true. You could provide a shortcut that configures the library via the command line but that should be optional and not the default. I agree with this. -Lars
Re: Implementing std.log
On Sun, 22 May 2011 21:38:26 +0300, Lars T. Kyllingstad public@kyllingen.nospamnet wrote: On Sun, 15 May 2011 17:15:38 +0200, Jacob Carlborg wrote: Why does the user have to manually initialize the library? Why not use a static constructor or lazy initialization? Actually I don't like it at all, that the logging library is configured via command line options. Seems very odd to me in the first place. That is something that should be handled by the application that uses std.log not the library itself. The library should be configurable via regular methods, like log.verbose = true. You could provide a shortcut that configures the library via the command line but that should be optional and not the default. I agree with this. -Lars I overall like the library, thanks for the work! Just a question, not only about std.log but upcoming projects as well. Is adopting to a library standard which was originally designed for another language any good? I think the library designers are influenced mostly by the capabilities of a language.
Re: Implementing std.log
Andrei Alexandrescu wrote: On 5/18/11 8:00 AM, Jose Armando Garcia wrote: I also think that having competing logging module is a good thing. This process will result in a better module for phobos and as a result a better module for the user. Clearly there are advantages to competing proposals, but I have mixed feelings about the whole thing, with the negative probably being stronger than the positive. At the end of the day one of us will have their work go to waste. Generally nobody wants their work to go to waste, and in particular I think I'd harm the community's interests by working on redundant stuff when so many other things are in need of attention. The reason I started std.log was out of desperation that everybody wanted to talk about it instead of working on it, and because I figured nobody would implement it the way I thought it needs to be done anyway. That's why I wrote on Apr 21st: Again, I'd _much_ rather prefer if someone just implemented this: http://google-glog.googlecode.com/svn/trunk/doc/glog.html It's simple, to the point, and brings the bacon home. In fact I'm putting dibs on this. I'll implement the thing and make a proposal. That message was meant to prevent exactly what's happening now. Honestly it hurts me that of all things possible, a new and talented contributor (two, to count Jacob's effort on his prototype) chose to work squarely on this one artifact. Right now the two APIs are converging and start differing in minor details, which makes it painfully obvious that at the end of the day two people are working in separation on virtually identical code. I can't afford this. I am ceasing work on std.log (feel free to copy from my code) and I encourage you to work towards a formal proposal. In doing that, I'll be upfront in saying that I'll very strongly advocate staying close to the client interface of std.log as it is now. In particular, every departure from glog (to which both designs now owe a lot) for equivalent functionality is gratuitous unless thoroughly justified. Yeah. If two proposals converge to one it makes no sense anymore. But I believe your proposal made std.log go into the very right direction. Sometimes it just better to have two proposals in very beginning to see what approach is better. But now there is no need for two proposals. Jose is heading in a better direction thanks to your proposal. In detail: * initializeLogging(SharedLogger.getCreator(args[0])); adds too much cognitive load, literally at line one. Now I need to know what a shared logger is and what a creator is. Just pass the entire command line and let the log subsystem pick its parameters. Don't use a delegate unless you can't do without. If client code wants to do configuration work, give them functions, don't ask them for a callback. The callback complicates things for no reason. I don't like it, too. It's too complicated. Meaning some people won't use it. It needs to be simple. * The name initializeLogging is a gratuitous departure from glog. Use initLogging. * Keep the glog flags as they are. In all likelihood any user of glog would want to define such flags, so we may as well spare them the work. * Pick the logging directory like glog does. * Add programmatic setting of flag names and values before calling initLogging. I think that's a good idea that doesn't depart from glog's design (google cmdline parser uses global names a la FLAGS_xxx). That would allow users to change a flag's name or disable it entirely (by assigning null to them). Assigning to a flag's value prior to calling initLogging would change its default. Assigning to a flag's value after initLogging forces the flag. To simply prevent log to get to any flags, client can call initLogging(args[0 .. 1]). Seems useful. * The obvious action to do with a log is to write to it. I don't want to write: log!info.write(stuff); I want to write: logInfo(stuff); or, fine, log!info(stuff); Agree. Saying write should be needless. * The names LOGGING_FATAL_DISABLED etc. are gratuitous departures from glog. Do what glog does adapted to D's naming convention, hence strip_log_xxx. * I don't want to write log!info(args.length 1).write(Arguments: , args[1 .. $]); Putting the condition there makes no sense without the manual. I want to write logInfo.when(args.length 1)(Arguments: , args[1 .. $]); or log!info.when(args.length 1)(Arguments: , args[1 .. $]); which omits the obvious write but adds the non-obvious when. So your point is than when should be explicit because it's not obvious. Seems right to me. Because if something gets more space (here saying when) it will get more attention. I think as was already said having the condition also in the log file should be helpful. We should try to have it in the log file. * Factoring out every, first, etc. is an interesting idea but I found no use for it outside logging.
Re: Implementing std.log
On 5/19/11 2:56 AM, Jens Mueller wrote: Andrei Alexandrescu wrote: * The way I see a nice implementation would be (inspired from Jens' work) via a class that defines the client-level methods as final, and has 2-3 extension methods that do the work. That way there's no need for awkward extra names (Logged/Logger, ouch) - one class encapsulates them all. I'm puzzled. I haven't done any work in that regard. Maybe you mean a different Jens. My inspiration (if existent) was little. It was Jacob. Apologies to both. Andrei
Re: Implementing std.log
On 2011-05-17 22:15, Andrei Alexandrescu wrote: On 5/17/11 4:02 AM, Jacob Carlborg wrote: On 2011-05-16 02:05, Andrei Alexandrescu wrote: Thanks for your work. I think there's an important distinction to be made. There are two APIs being discussed. One is the client interface and the other is the extensibility interface. Jose looked into both: he provided a client interface that has formatting, levels, enabling, and such, and an extensibility interface that essentially is a simple output stream. My library explores the client interface and leaves the extensibility interface as an obvious piece of work that needs little agreement and minimal design effort. Finally, your library keeps the client interface to a minimum and focuses almost exclusively on the extensibility interface. In doing so, it makes few choices that I disagree with. Allow me to share some specific feedback. Note that my suggestion was just a simple and incomplete suggestion on how the API could look like. I only provided info, warning and error methods as examples, I'm not saying the API should only have these three levels. [snip] I thought about this some more and I understand I sounded unfair. There is a lot of merit and there are a lot of good ideas in your code (and of course Jose's), which I didn't mention for the simple but cold reason that negative feedback is more informative. But neglecting the merits is a mistake as well. I'll incorporate some of the ideas you suggested in the next pass through std.log. Thanks, Andrei No hard feelings, I also have a tendency to just give negative feedback. -- /Jacob Carlborg
Re: Implementing std.log
std.logging is still alive! After posting my first attempt at a logging module I went back and spent a lot of time on how I could improve the API. I would like to say that Andrei's std.log module had a great influence on the final outcome. There are some aspect of std.log's API that I really like. E.g. the ability to do fine grain compile time configuration. I'll go into detail in a subsequent review of std.log. I also think that having competing logging module is a good thing. This process will result in a better module for phobos and as a result a better module for the user. The best way to see what has changed is to look at the source code (https://github.com/jsancio/phobos/blob/master/std/logging.d) and the generated doc (http://jsancio.github.com/phobos/phobos/std_logging.html). Here are some highlights: 1) I tried to minimize the amount of code the compiler has to generate because of template. Template are only used when the module has/wants to make compiled time decision. If you spot places where templates are not needed please tell me. 2) Make logging decision as early as possible. Logging can be short circuited by either the severity level, the verbose level or an user defined condition. All those conditions are evaluated as soon as possible and only when required. E.g. the module doesn't evaluate user define condition if it already knows that it wont log because of the severity. 3) A module should do one thing an do it well. This module does logging so unlike std.log and glog it doesn't have built in mechanism for doing 'every', 'when', 'first', etc. Instead the log and vlog function provide a lazily evaluated bool parameter that the user can use to perform condition logging. There are plenty of example of this in the doc. At the top of the doc I provided an example similar to std.log's synopsis so you can compare the two APIs. Disclaimer: This is a draft. Once we agree on the main API (log and vlog). I will go back an finish the implementation, test and documentation. Enjoy! Let me know if you have any suggestions, -Jose On Wed, May 18, 2011 at 6:05 AM, Jacob Carlborg d...@me.com wrote: On 2011-05-17 22:15, Andrei Alexandrescu wrote: On 5/17/11 4:02 AM, Jacob Carlborg wrote: On 2011-05-16 02:05, Andrei Alexandrescu wrote: Thanks for your work. I think there's an important distinction to be made. There are two APIs being discussed. One is the client interface and the other is the extensibility interface. Jose looked into both: he provided a client interface that has formatting, levels, enabling, and such, and an extensibility interface that essentially is a simple output stream. My library explores the client interface and leaves the extensibility interface as an obvious piece of work that needs little agreement and minimal design effort. Finally, your library keeps the client interface to a minimum and focuses almost exclusively on the extensibility interface. In doing so, it makes few choices that I disagree with. Allow me to share some specific feedback. Note that my suggestion was just a simple and incomplete suggestion on how the API could look like. I only provided info, warning and error methods as examples, I'm not saying the API should only have these three levels. [snip] I thought about this some more and I understand I sounded unfair. There is a lot of merit and there are a lot of good ideas in your code (and of course Jose's), which I didn't mention for the simple but cold reason that negative feedback is more informative. But neglecting the merits is a mistake as well. I'll incorporate some of the ideas you suggested in the next pass through std.log. Thanks, Andrei No hard feelings, I also have a tendency to just give negative feedback. -- /Jacob Carlborg
Re: Implementing std.log
This is a review of std.log. Overall, I really like the API exposed by this module because it allows efficient compile time and run time configuration. I have limited the review to the API and how the API affects the implementation. I will review the implementation once the API is close to final. 1) I think this was pointed out before. I don't really like that the module can only be configure through the command line by passing the list of command line arguments. The user should be able to configure the module programmatically. 2) Not a fan of the top level symbols logInfo, logFatal, etc but it is not a big deal. 3) opCall and format used template arguments for file and line yet they are never used at compile time. This leads to unnecessary template instantiations. 4) I know that when() and every() are really cool features but should they be first class citizens in a logging module? What if the user wants 'whenTheMoonIsFull(country)'? Including as part of the module could be better argued if std.log generates special messages for them which is what I think glog does. I have been thinking of ways of doing this in std.logging. E.g. 'logInfo.every(2)(Message)' would generate the following log line ... 2nd call ... Message. 5) General thought that applies to both std.log and std.logging: It would be interesting to allow some configuration after initialization. For example it would be unreasonable to allow configuration of things that were already externalized like the name of the log file. But maybe the user should be able to change the severity level and vlog configuration. Overall it looks good. As I said before I have a lot of comments on the implementation but will hold them until we have a final API. Thanks, -Jose On Wed, May 18, 2011 at 10:00 AM, Jose Armando Garcia jsan...@gmail.com wrote: std.logging is still alive! After posting my first attempt at a logging module I went back and spent a lot of time on how I could improve the API. I would like to say that Andrei's std.log module had a great influence on the final outcome. There are some aspect of std.log's API that I really like. E.g. the ability to do fine grain compile time configuration. I'll go into detail in a subsequent review of std.log. I also think that having competing logging module is a good thing. This process will result in a better module for phobos and as a result a better module for the user. The best way to see what has changed is to look at the source code (https://github.com/jsancio/phobos/blob/master/std/logging.d) and the generated doc (http://jsancio.github.com/phobos/phobos/std_logging.html). Here are some highlights: 1) I tried to minimize the amount of code the compiler has to generate because of template. Template are only used when the module has/wants to make compiled time decision. If you spot places where templates are not needed please tell me. 2) Make logging decision as early as possible. Logging can be short circuited by either the severity level, the verbose level or an user defined condition. All those conditions are evaluated as soon as possible and only when required. E.g. the module doesn't evaluate user define condition if it already knows that it wont log because of the severity. 3) A module should do one thing an do it well. This module does logging so unlike std.log and glog it doesn't have built in mechanism for doing 'every', 'when', 'first', etc. Instead the log and vlog function provide a lazily evaluated bool parameter that the user can use to perform condition logging. There are plenty of example of this in the doc. At the top of the doc I provided an example similar to std.log's synopsis so you can compare the two APIs. Disclaimer: This is a draft. Once we agree on the main API (log and vlog). I will go back an finish the implementation, test and documentation. Enjoy! Let me know if you have any suggestions, -Jose On Wed, May 18, 2011 at 6:05 AM, Jacob Carlborg d...@me.com wrote: On 2011-05-17 22:15, Andrei Alexandrescu wrote: On 5/17/11 4:02 AM, Jacob Carlborg wrote: On 2011-05-16 02:05, Andrei Alexandrescu wrote: Thanks for your work. I think there's an important distinction to be made. There are two APIs being discussed. One is the client interface and the other is the extensibility interface. Jose looked into both: he provided a client interface that has formatting, levels, enabling, and such, and an extensibility interface that essentially is a simple output stream. My library explores the client interface and leaves the extensibility interface as an obvious piece of work that needs little agreement and minimal design effort. Finally, your library keeps the client interface to a minimum and focuses almost exclusively on the extensibility interface. In doing so, it makes few choices that I disagree with. Allow me to share some specific feedback. Note that my suggestion was just a simple and incomplete suggestion on
Re: Implementing std.log
On 5/18/11 8:00 AM, Jose Armando Garcia wrote:
I also think that having competing logging module is a good thing.
This process will result in a better module for phobos and as a result
a better module for the user.
Clearly there are advantages to competing proposals, but I have mixed
feelings about the whole thing, with the negative probably being
stronger than the positive.
At the end of the day one of us will have their work go to waste.
Generally nobody wants their work to go to waste, and in particular I
think I'd harm the community's interests by working on redundant stuff
when so many other things are in need of attention. The reason I started
std.log was out of desperation that everybody wanted to talk about it
instead of working on it, and because I figured nobody would implement
it the way I thought it needs to be done anyway. That's why I wrote on
Apr 21st:
Again, I'd _much_ rather prefer if someone just implemented this:
http://google-glog.googlecode.com/svn/trunk/doc/glog.html
It's simple, to the point, and brings the bacon home.
In fact I'm putting dibs on this. I'll implement the thing and make a
proposal.
That message was meant to prevent exactly what's happening now. Honestly
it hurts me that of all things possible, a new and talented contributor
(two, to count Jacob's effort on his prototype) chose to work squarely
on this one artifact.
Right now the two APIs are converging and start differing in minor
details, which makes it painfully obvious that at the end of the day two
people are working in separation on virtually identical code. I can't
afford this.
I am ceasing work on std.log (feel free to copy from my code) and I
encourage you to work towards a formal proposal. In doing that, I'll be
upfront in saying that I'll very strongly advocate staying close to the
client interface of std.log as it is now. In particular, every departure
from glog (to which both designs now owe a lot) for equivalent
functionality is gratuitous unless thoroughly justified. In detail:
* initializeLogging(SharedLogger.getCreator(args[0])); adds too much
cognitive load, literally at line one. Now I need to know what a shared
logger is and what a creator is. Just pass the entire command line and
let the log subsystem pick its parameters. Don't use a delegate unless
you can't do without. If client code wants to do configuration work,
give them functions, don't ask them for a callback. The callback
complicates things for no reason.
* The name initializeLogging is a gratuitous departure from glog. Use
initLogging.
* Keep the glog flags as they are. In all likelihood any user of glog
would want to define such flags, so we may as well spare them the work.
* Pick the logging directory like glog does.
* Add programmatic setting of flag names and values before calling
initLogging. I think that's a good idea that doesn't depart from glog's
design (google cmdline parser uses global names a la FLAGS_xxx). That
would allow users to change a flag's name or disable it entirely (by
assigning null to them). Assigning to a flag's value prior to calling
initLogging would change its default. Assigning to a flag's value after
initLogging forces the flag. To simply prevent log to get to any flags,
client can call initLogging(args[0 .. 1]).
* The obvious action to do with a log is to write to it. I don't want to
write:
log!info.write(stuff);
I want to write:
logInfo(stuff);
or, fine,
log!info(stuff);
* The names LOGGING_FATAL_DISABLED etc. are gratuitous departures from
glog. Do what glog does adapted to D's naming convention, hence
strip_log_xxx.
* I don't want to write
log!info(args.length 1).write(Arguments: , args[1 .. $]);
Putting the condition there makes no sense without the manual. I want to
write
logInfo.when(args.length 1)(Arguments: , args[1 .. $]);
or
log!info.when(args.length 1)(Arguments: , args[1 .. $]);
which omits the obvious write but adds the non-obvious when.
* Factoring out every, first, etc. is an interesting idea but I found no
use for it outside logging. (That doesn't mean there isn't any, it just
means we should think of it because cool things may happen.) That
shouldn't harm except when combined with the point above we're forced to:
logInfo.when(every(1000))(text);
which is self-explanatory but rather verbose. In all honesty
log!info(every(9)).write(Every nine); isn't that easy on the eyes either.
* Define after in addition to every and first, and overload them
all for core.Duration. It's rather simple, for example for every you'd
have something like:
static ulong lastTimeInHnsecs;
immutable ulong now = Clock.currTime.stdTime;
if (dur!hnsecs(now - lastTimeInHnsecs) d)
{
return nullLogger; // no logging this time
}
lastTimeInHnsecs = now;
return this; // will log
* I peeked at the implementation and you allocate one new string for
each logged message. You must keep a buffer in thread-local
Re: Implementing std.log
On 2011-05-16 01:04, Andrei Alexandrescu wrote: On 05/15/2011 10:22 AM, Jacob Carlborg wrote: On 2011-05-14 22:37, Andrei Alexandrescu wrote: I want to nail usability before extension. Extensibility can be provided via an indirection inside FileLogger (which should indeed receive a more appropriate name). Andrei The obvious solution would be to have the logging functions returning an interface, Logger, but interfaces seem to be banned from Phobos. Ok, I see now, since templates are used this can be a problem. Hmm, templates are used EVERYWHERE in Phobos. No need to be ironic. Interfaces are the obvious solution, and what I meant when I said indirection (i.e. via a reference type, either interface or class). That being said, I think your design has a number of issues that I'll get into soon. Andrei It wasn't my intention to be ironic. I missed indirection and just read something about inside FileLogger. -- /Jacob Carlborg
Re: Implementing std.log
On 2011-05-16 01:27, Jonathan M Davis wrote: On 2011-05-15 16:01, Andrei Alexandrescu wrote: On 05/15/2011 10:26 AM, Jacob Carlborg wrote: On 2011-05-15 00:33, Andrei Alexandrescu wrote: On 05/14/2011 05:17 PM, Jonathan M Davis wrote: On 2011-05-14 14:56, Michel Fortin wrote: On 2011-05-14 17:31:30 -0400, Jonathan M Davisjmdavisp...@gmx.com said: So, I do think that knowing which thread is logging what could be very important for some programs, but I don't think that separating the log files is necessarily a good idea. If you did, you'd lose timing information (unless the time is at the beginning of every log line (which could also be useful), but then you'd have to read the times and compare them to see what happened before what). So, I'd be all for some options and extra information which could be added to each log line which would help debugging, but I don't think that thread-local logs is a great idea. I'd even go further and question whether it makes sense to have info, warning, and errors be written to separate files. I'd definitely vote for them all to be in the same file, but I don't generally see much benefit in having multiple log files. I like having them all in one place where you can see what happened in what order. Having them in separate log files is just going to make it harder to figure out what happened, and I think that it would become tempting (for me at least) to just log everything at exactly the same level so that they ended up in the same file. The info log contains log messages for all levels. Generally I'd want to stray from glog's major design decision as little as possible. It's an approach validated by years of experience in heavy-duty applications. Also, people coming from glog and used to its features would find it unpleasant that we decided to do things a different way without solid reasons. Andrei Why not? It doesn't say anywhere in the documentation, as far as I can see, that it's based on glog. That's beside the point as the documentation could and should be changed. I've said repeatedly it's based on glog, and I think it's a good thing. I think that, in general, that is a poor argument: Because library A does it like that we also have to do it like that. What about all people coming from library B, C, D and so on, it will be unpleasant for those as well because we didn't follow their particular library design. It's a good argument if library A is considered better by the person making the argument than B, C, and D. One thing is that glog has most desirable qualities I am looking for in a logging library: allows compile-time and run-time enabling/disabling, it's relatively small, it's simple, it's effective, and is widely used. I'm sure there are other libraries that score better on either dimension, but not on all. There are two departures I chose to make from glog: (a) I added the critical log which throws an exceptions (Google wouldn't need one because they don't use exceptions), and (b) I replaced the clunky LOG_IF_EVERY_N etc. with the flexible when(), every(), after() etc. I think these are well justified choices. I think that it makes good sense to start with an API based on an existing API which is known and heavily used and then adjust it as appropriate to our needs. In doing so, we might as well leave it the same as the original where we don't need to change it, since there is some gain for those familiar with the original. However, the real question then is where it does and doesn't make sense to change it. If enough people agree with Andrei on those decisions, then it will likely become std.log. However, if there's enough disagreement, and we can't reach an appropriate compromise on such changes, then it likely won't become std.log. Hopefully we can come a solid design based on glog with whatever changes we actually need without having to drastically change what it does in comparison to glog. That would be ideal. However, I also think that if it really looks like the general consensus is against something about how glog does things, we shouldn't be married to how glog works. So, overall, I agree with Andrei. The devil is the details, however. - Jonathan M Davis I agree with Jonathan, if glog is good enough we can follow its design but if it isn't, we shouldn't hesitate to make some design changes. The approach I take when designing an API for a library I'm writing I look at the API of several other libraries that does the same thing. Then I try to take the best parts and the parts I like of each library and combine them into something that make sense. -- /Jacob Carlborg
Re: Implementing std.log
On 2011-05-16 02:05, Andrei Alexandrescu wrote: Thanks for your work. I think there's an important distinction to be made. There are two APIs being discussed. One is the client interface and the other is the extensibility interface. Jose looked into both: he provided a client interface that has formatting, levels, enabling, and such, and an extensibility interface that essentially is a simple output stream. My library explores the client interface and leaves the extensibility interface as an obvious piece of work that needs little agreement and minimal design effort. Finally, your library keeps the client interface to a minimum and focuses almost exclusively on the extensibility interface. In doing so, it makes few choices that I disagree with. Allow me to share some specific feedback. Note that my suggestion was just a simple and incomplete suggestion on how the API could look like. I only provided info, warning and error methods as examples, I'm not saying the API should only have these three levels. 1. It becomes dynamic too early. First, in order to disable logging during compilation, a barrier of aliased types is needed. The alias would choose either a null type that does nothing (see StaticNullLogger at https://github.com/andralex/phobos/blob/master/std/log.d), or a log that actually does something. I think your design can be adjusted to do that with relative ease. I didn't consider disabling logging during compilation at all. If people like the basic idea with my API then, of course, I would flesh it out and add support compile time disabling of the logging. 2. It becomes dynamic too early from a different perspective. The interface for extensibility is identical with the interface for use and as a consequence leaves too broad functionality unimplemented. A better design is to handle a bunch of decisions (is the log dynamically enabled? every N calls? every N seconds? before N seconds have passed? after N seconds have passed?) in a lightweight front end, i.e. _before_ arguments have been evaluated (this is crucial!). Also, the front end should take care of formatting minutiae, leaving the dynamic back-end the simple task of streaming text where it's supposed to be going. I think Jose's design did the right thing there by doing enabled checks and formatting in the front-end, leaving only transport to the back-end. 3. Another consequence is that extensibility points have the signature: Logger log(string file, long line, ...); which leaves the heavyweight formatting task to the antiquated ... interface (no static types left, meaning e.g. no structs defining toString can be logged). Even assuming that's not a problem, implementors of this function are left with a fair amount of work to do. The problem with variadic template methods are that they're not virtual. 4. The Logger class contains info, warning, and error Level objects. Each Level object in turn contains a reference to a Logger. This design leaves some odd shrapnel behind, e.g. one can get to the critical log from the info log. I thought it was a good idea that you could chain calls like this: log.info(foo).error(bar); 5. I've seen the notion of a default logging level elsewhere but I think it's inadequate for the info/warning/error/critical/fatal approach. It is adequate for verbosity levels, i.e. vlog and friends. For info/warning/etc. the decision of the appropriate level belongs to the caller. Ok. Just a note, nobody is forcing you to use the default logging level. 6. My initial design also contained a global log battery containing the info, warning, etc. objects as members. I found that design stilted because it provides absolutely no benefit besides a marginal log.info vs. logInfo, has no encapsulation advantage, is adverse to extensibility, and worse of all perpetuates poor object oriented design. It's best to call a spade a spade, so there should be five global objects because that's in keep with reality. Andrei I got the impression that you change log.info to logInfo just because you wanted user defined loggers to look the same. My suggestion shows that's not necessary. Also I tried to minimize the use of global variables. -- /Jacob Carlborg
Re: Implementing std.log
On 5/17/11 4:02 AM, Jacob Carlborg wrote: On 2011-05-16 02:05, Andrei Alexandrescu wrote: Thanks for your work. I think there's an important distinction to be made. There are two APIs being discussed. One is the client interface and the other is the extensibility interface. Jose looked into both: he provided a client interface that has formatting, levels, enabling, and such, and an extensibility interface that essentially is a simple output stream. My library explores the client interface and leaves the extensibility interface as an obvious piece of work that needs little agreement and minimal design effort. Finally, your library keeps the client interface to a minimum and focuses almost exclusively on the extensibility interface. In doing so, it makes few choices that I disagree with. Allow me to share some specific feedback. Note that my suggestion was just a simple and incomplete suggestion on how the API could look like. I only provided info, warning and error methods as examples, I'm not saying the API should only have these three levels. [snip] I thought about this some more and I understand I sounded unfair. There is a lot of merit and there are a lot of good ideas in your code (and of course Jose's), which I didn't mention for the simple but cold reason that negative feedback is more informative. But neglecting the merits is a mistake as well. I'll incorporate some of the ideas you suggested in the next pass through std.log. Thanks, Andrei
Re: Implementing std.log
On 14/05/11 19.04, Andrei Alexandrescu wrote: On 5/9/11 1:52 AM, Andrei Alexandrescu wrote: [snip] I updated my std.log draft. Added a lot of features including formatted writing, delayed logging, and a variety of configuration options. Replaced the redundant log.xyz with logXyz. The implementation is getting close to reviewable form. Documentation: http://d-programming-language.org/phobos-prerelease/std_log.html Source: https://github.com/andralex/phobos Feedback welcome. I like it! Some minor suggestions: The command line flags seem inconsistent in their naming e.g --minloglevel vs. --log_dir (the latter using underscores). Also single letter flags are usually written with a single '-' in front instead of '--' at least on posix platforms. /Jonas
Re: Implementing std.log
Andrei Alexandrescu wrote: On 5/9/11 1:52 AM, Andrei Alexandrescu wrote: [snip] I updated my std.log draft. Added a lot of features including formatted writing, delayed logging, and a variety of configuration options. Replaced the redundant log.xyz with logXyz. The implementation is getting close to reviewable form. Documentation: http://d-programming-language.org/phobos-prerelease/std_log.html Source: https://github.com/andralex/phobos Feedback welcome. I started to test it out. Some things to consider: * If I miss a minus on a program option that option will be silently missed. E.g. I wrote -minloglevel 0 instead of --minloglevel 0 and was quite puzzled that I didn't get all messages. * Rename TEST_TMPDIR to LOG_TMPDIR or LOG_DIR? * I believe printing a ulong as thread id is too wasteful and makes the output harder to read. * I would arrange the loggers in increasing order of importance in the documentation (and source), i.e. logInfo, logWarning, ... And at very last: vlog. In the same order as done in the synopsis. * I'm unsure whether the logging levels should better be implemented using enum instead of dchars[]. Because 'I', 'W', 'E', 'C', 'F' are used several times in the source. * Instead of logInfo, logWarning, logError etc. just naming them info, warn, error etc. may be an option. I think it's quite obvious that these are logging statements. No need for the prefix log? * Even though vlog is short maybe logVerbose is more consistent. But admittedly it's quite long. I'm trying to read linker map files to evaluate how much code is generated when using the strip_log_* version options. Is there some good documentation on how to read those files? I believe I will have to read some basic literature about linker and loaders. Jens
Re: Implementing std.log
On 2011-05-14 19:04, Andrei Alexandrescu wrote: On 5/9/11 1:52 AM, Andrei Alexandrescu wrote: [snip] I updated my std.log draft. Added a lot of features including formatted writing, delayed logging, and a variety of configuration options. Replaced the redundant log.xyz with logXyz. The implementation is getting close to reviewable form. Documentation: http://d-programming-language.org/phobos-prerelease/std_log.html Source: https://github.com/andralex/phobos Feedback welcome. Thanks, Andrei Why does the user have to manually initialize the library? Why not use a static constructor or lazy initialization? Actually I don't like it at all, that the logging library is configured via command line options. Seems very odd to me in the first place. That is something that should be handled by the application that uses std.log not the library itself. The library should be configurable via regular methods, like log.verbose = true. You could provide a shortcut that configures the library via the command line but that should be optional and not the default. -- /Jacob Carlborg
Re: Implementing std.log
On 2011-05-14 22:37, Andrei Alexandrescu wrote: On 05/14/2011 02:50 PM, Robert Clipsham wrote: On 14/05/2011 18:04, Andrei Alexandrescu wrote: On 5/9/11 1:52 AM, Andrei Alexandrescu wrote: [snip] I updated my std.log draft. Added a lot of features including formatted writing, delayed logging, and a variety of configuration options. Replaced the redundant log.xyz with logXyz. The implementation is getting close to reviewable form. Documentation: http://d-programming-language.org/phobos-prerelease/std_log.html Source: https://github.com/andralex/phobos Feedback welcome. Thanks, Andrei I far prefered log.xyz to logXyz... The latter just looks ugly in my opinion. There are a few problems with log.xyz. For one, std.log.log.info is terrible. Second, I found absolutely no justification to provide the dummy struct log just to keep five predefined logs in there. Them being individual entities allows the user to define their own logs with the same look and feel, e.g. logRemote. A also notice all those functions return FileLoggers, how will this work when other backends are supported? I want to nail usability before extension. Extensibility can be provided via an indirection inside FileLogger (which should indeed receive a more appropriate name). Andrei The obvious solution would be to have the logging functions returning an interface, Logger, but interfaces seem to be banned from Phobos. Ok, I see now, since templates are used this can be a problem. Hmm, templates are used EVERYWHERE in Phobos. -- /Jacob Carlborg
Re: Implementing std.log
On 2011-05-15 00:33, Andrei Alexandrescu wrote: On 05/14/2011 05:17 PM, Jonathan M Davis wrote: On 2011-05-14 14:56, Michel Fortin wrote: On 2011-05-14 17:31:30 -0400, Jonathan M Davisjmdavisp...@gmx.com said: So, I do think that knowing which thread is logging what could be very important for some programs, but I don't think that separating the log files is necessarily a good idea. If you did, you'd lose timing information (unless the time is at the beginning of every log line (which could also be useful), but then you'd have to read the times and compare them to see what happened before what). So, I'd be all for some options and extra information which could be added to each log line which would help debugging, but I don't think that thread-local logs is a great idea. I'd even go further and question whether it makes sense to have info, warning, and errors be written to separate files. I'd definitely vote for them all to be in the same file, but I don't generally see much benefit in having multiple log files. I like having them all in one place where you can see what happened in what order. Having them in separate log files is just going to make it harder to figure out what happened, and I think that it would become tempting (for me at least) to just log everything at exactly the same level so that they ended up in the same file. The info log contains log messages for all levels. Generally I'd want to stray from glog's major design decision as little as possible. It's an approach validated by years of experience in heavy-duty applications. Also, people coming from glog and used to its features would find it unpleasant that we decided to do things a different way without solid reasons. Andrei Why not? It doesn't say anywhere in the documentation, as far as I can see, that it's based on glog. I think that, in general, that is a poor argument: Because library A does it like that we also have to do it like that. What about all people coming from library B, C, D and so on, it will be unpleasant for those as well because we didn't follow their particular library design. -- /Jacob Carlborg
Re: Implementing std.log
On 15/05/2011 16:15, Jacob Carlborg wrote: On 2011-05-14 19:04, Andrei Alexandrescu wrote: On 5/9/11 1:52 AM, Andrei Alexandrescu wrote: [snip] I updated my std.log draft. Added a lot of features including formatted writing, delayed logging, and a variety of configuration options. Replaced the redundant log.xyz with logXyz. The implementation is getting close to reviewable form. Documentation: http://d-programming-language.org/phobos-prerelease/std_log.html Source: https://github.com/andralex/phobos Feedback welcome. Thanks, Andrei Why does the user have to manually initialize the library? Why not use a static constructor or lazy initialization? Actually I don't like it at all, that the logging library is configured via command line options. Seems very odd to me in the first place. That is something that should be handled by the application that uses std.log not the library itself. The library should be configurable via regular methods, like log.verbose = true. You could provide a shortcut that configures the library via the command line but that should be optional and not the default. Agreed. -- Robert http://octarineparrot.com/
Re: Implementing std.log
Andrei Alexandrescu wrote:
On 05/14/2011 04:56 PM, Michel Fortin wrote:
On 2011-05-14 17:31:30 -0400, Jonathan M Davis jmdavisp...@gmx.com said:
So, I do think that knowing which thread is logging what could be very
important for some programs, but I don't think that separating the log
files
is necessarily a good idea. If you did, you'd lose timing information
(unless
the time is at the beginning of every log line (which could also be
useful),
but then you'd have to read the times and compare them to see what
happened
before what). So, I'd be all for some options and extra information which
could be added to each log line which would help debugging, but I
don't think
that thread-local logs is a great idea.
I'd even go further and question whether it makes sense to have info,
warning, and errors be written to separate files.
I'll also question whether they should be written to files at all by
default (as opposed to stdin and stderr). I'm aware initLogging's
documentation says: If logÂging is efÂfected withÂout havÂing called
this funcÂtion, all paÂraÂmeÂters are at their deÂfault valÂues and all
logÂging is done only to stderr. So basically, I'll get what I want if
I never call initLogging, but then I can't control verbosity and other
settings using --v and other flags passed as arguments.
A server app must log to files, no question about that. We'll need
to add rotation etc. in the future. But I do plan to offer ways to
manually override defaults, e.g.:
void main(string[] args)
{
logtostderr = true; // override default
initLogging(args);
...
}
How do you plan to do this?
Because it looks partly what gflags is doing. At least it would be
useful as general feature for handling program options.
I haven't found a way to define flags in a compilation unit as gflags
does. I have code that generates a help string and even reading flags
from a files is easy. But I have no clue yet how to allow declaring
flags in different compilation units. Technically I'd like to allow each
flag that can be handled by std.getopt but I have no idea how to check
whether getopt supports a given flag. If I knew I'd store flags in
associative array where values are Algebratic!(GetoptFlagTypes).
Jens
Re: Implementing std.log
On 2011-05-14 19:04, Andrei Alexandrescu wrote: On 5/9/11 1:52 AM, Andrei Alexandrescu wrote: [snip] I updated my std.log draft. Added a lot of features including formatted writing, delayed logging, and a variety of configuration options. Replaced the redundant log.xyz with logXyz. The implementation is getting close to reviewable form. Documentation: http://d-programming-language.org/phobos-prerelease/std_log.html Source: https://github.com/andralex/phobos Feedback welcome. Thanks, Andrei How about an API that looks something like this: http://pastebin.com/dLVp1GRr This API contains standard interfaces for a logger and a logging level. It contains a default logger implementation and a couple of logging levels. This API also allows you to implement custom loggers and custom logging levels. Multiple logger instances are possible with options set on each instance. Note this is just a suggestion for the API and doesn't contain any real implementation (just prints to stdout). -- /Jacob Carlborg
Re: Implementing std.log
On 05/15/2011 10:53 AM, Jens Mueller wrote:
Andrei Alexandrescu wrote:
On 05/14/2011 04:56 PM, Michel Fortin wrote:
On 2011-05-14 17:31:30 -0400, Jonathan M Davisjmdavisp...@gmx.com said:
So, I do think that knowing which thread is logging what could be very
important for some programs, but I don't think that separating the log
files
is necessarily a good idea. If you did, you'd lose timing information
(unless
the time is at the beginning of every log line (which could also be
useful),
but then you'd have to read the times and compare them to see what
happened
before what). So, I'd be all for some options and extra information which
could be added to each log line which would help debugging, but I
don't think
that thread-local logs is a great idea.
I'd even go further and question whether it makes sense to have info,
warning, and errors be written to separate files.
I'll also question whether they should be written to files at all by
default (as opposed to stdin and stderr). I'm aware initLogging's
documentation says: If logÂging is efÂfected withÂout havÂing called
this funcÂtion, all paÂraÂmeÂters are at their deÂfault valÂues and all
logÂging is done only to stderr. So basically, I'll get what I want if
I never call initLogging, but then I can't control verbosity and other
settings using --v and other flags passed as arguments.
A server app must log to files, no question about that. We'll need
to add rotation etc. in the future. But I do plan to offer ways to
manually override defaults, e.g.:
void main(string[] args)
{
logtostderr = true; // override default
initLogging(args);
...
}
How do you plan to do this?
Because it looks partly what gflags is doing. At least it would be
useful as general feature for handling program options.
I haven't found a way to define flags in a compilation unit as gflags
does. I have code that generates a help string and even reading flags
from a files is easy. But I have no clue yet how to allow declaring
flags in different compilation units. Technically I'd like to allow each
flag that can be handled by std.getopt but I have no idea how to check
whether getopt supports a given flag. If I knew I'd store flags in
associative array where values are Algebratic!(GetoptFlagTypes).
Jens
The way the Google flags library does that is by planting registration
functions globally. Then initGoogle(argc, argv) at the beginning of main
takes care of calling all registration functions.
We could add such a hooking capability to getopt, but the current
std.log design doesn't need one - it just calls getopt itself and cherry
picks the parameters it's interested in.
Andrei
Re: Implementing std.log
On 05/15/2011 10:26 AM, Jacob Carlborg wrote: On 2011-05-15 00:33, Andrei Alexandrescu wrote: On 05/14/2011 05:17 PM, Jonathan M Davis wrote: On 2011-05-14 14:56, Michel Fortin wrote: On 2011-05-14 17:31:30 -0400, Jonathan M Davisjmdavisp...@gmx.com said: So, I do think that knowing which thread is logging what could be very important for some programs, but I don't think that separating the log files is necessarily a good idea. If you did, you'd lose timing information (unless the time is at the beginning of every log line (which could also be useful), but then you'd have to read the times and compare them to see what happened before what). So, I'd be all for some options and extra information which could be added to each log line which would help debugging, but I don't think that thread-local logs is a great idea. I'd even go further and question whether it makes sense to have info, warning, and errors be written to separate files. I'd definitely vote for them all to be in the same file, but I don't generally see much benefit in having multiple log files. I like having them all in one place where you can see what happened in what order. Having them in separate log files is just going to make it harder to figure out what happened, and I think that it would become tempting (for me at least) to just log everything at exactly the same level so that they ended up in the same file. The info log contains log messages for all levels. Generally I'd want to stray from glog's major design decision as little as possible. It's an approach validated by years of experience in heavy-duty applications. Also, people coming from glog and used to its features would find it unpleasant that we decided to do things a different way without solid reasons. Andrei Why not? It doesn't say anywhere in the documentation, as far as I can see, that it's based on glog. That's beside the point as the documentation could and should be changed. I've said repeatedly it's based on glog, and I think it's a good thing. I think that, in general, that is a poor argument: Because library A does it like that we also have to do it like that. What about all people coming from library B, C, D and so on, it will be unpleasant for those as well because we didn't follow their particular library design. It's a good argument if library A is considered better by the person making the argument than B, C, and D. One thing is that glog has most desirable qualities I am looking for in a logging library: allows compile-time and run-time enabling/disabling, it's relatively small, it's simple, it's effective, and is widely used. I'm sure there are other libraries that score better on either dimension, but not on all. There are two departures I chose to make from glog: (a) I added the critical log which throws an exceptions (Google wouldn't need one because they don't use exceptions), and (b) I replaced the clunky LOG_IF_EVERY_N etc. with the flexible when(), every(), after() etc. I think these are well justified choices. Andrei
Re: Implementing std.log
On 05/15/2011 10:22 AM, Jacob Carlborg wrote: On 2011-05-14 22:37, Andrei Alexandrescu wrote: I want to nail usability before extension. Extensibility can be provided via an indirection inside FileLogger (which should indeed receive a more appropriate name). Andrei The obvious solution would be to have the logging functions returning an interface, Logger, but interfaces seem to be banned from Phobos. Ok, I see now, since templates are used this can be a problem. Hmm, templates are used EVERYWHERE in Phobos. No need to be ironic. Interfaces are the obvious solution, and what I meant when I said indirection (i.e. via a reference type, either interface or class). That being said, I think your design has a number of issues that I'll get into soon. Andrei
Re: Implementing std.log
On 2011-05-15 16:01, Andrei Alexandrescu wrote: On 05/15/2011 10:26 AM, Jacob Carlborg wrote: On 2011-05-15 00:33, Andrei Alexandrescu wrote: On 05/14/2011 05:17 PM, Jonathan M Davis wrote: On 2011-05-14 14:56, Michel Fortin wrote: On 2011-05-14 17:31:30 -0400, Jonathan M Davisjmdavisp...@gmx.com said: So, I do think that knowing which thread is logging what could be very important for some programs, but I don't think that separating the log files is necessarily a good idea. If you did, you'd lose timing information (unless the time is at the beginning of every log line (which could also be useful), but then you'd have to read the times and compare them to see what happened before what). So, I'd be all for some options and extra information which could be added to each log line which would help debugging, but I don't think that thread-local logs is a great idea. I'd even go further and question whether it makes sense to have info, warning, and errors be written to separate files. I'd definitely vote for them all to be in the same file, but I don't generally see much benefit in having multiple log files. I like having them all in one place where you can see what happened in what order. Having them in separate log files is just going to make it harder to figure out what happened, and I think that it would become tempting (for me at least) to just log everything at exactly the same level so that they ended up in the same file. The info log contains log messages for all levels. Generally I'd want to stray from glog's major design decision as little as possible. It's an approach validated by years of experience in heavy-duty applications. Also, people coming from glog and used to its features would find it unpleasant that we decided to do things a different way without solid reasons. Andrei Why not? It doesn't say anywhere in the documentation, as far as I can see, that it's based on glog. That's beside the point as the documentation could and should be changed. I've said repeatedly it's based on glog, and I think it's a good thing. I think that, in general, that is a poor argument: Because library A does it like that we also have to do it like that. What about all people coming from library B, C, D and so on, it will be unpleasant for those as well because we didn't follow their particular library design. It's a good argument if library A is considered better by the person making the argument than B, C, and D. One thing is that glog has most desirable qualities I am looking for in a logging library: allows compile-time and run-time enabling/disabling, it's relatively small, it's simple, it's effective, and is widely used. I'm sure there are other libraries that score better on either dimension, but not on all. There are two departures I chose to make from glog: (a) I added the critical log which throws an exceptions (Google wouldn't need one because they don't use exceptions), and (b) I replaced the clunky LOG_IF_EVERY_N etc. with the flexible when(), every(), after() etc. I think these are well justified choices. I think that it makes good sense to start with an API based on an existing API which is known and heavily used and then adjust it as appropriate to our needs. In doing so, we might as well leave it the same as the original where we don't need to change it, since there is some gain for those familiar with the original. However, the real question then is where it does and doesn't make sense to change it. If enough people agree with Andrei on those decisions, then it will likely become std.log. However, if there's enough disagreement, and we can't reach an appropriate compromise on such changes, then it likely won't become std.log. Hopefully we can come a solid design based on glog with whatever changes we actually need without having to drastically change what it does in comparison to glog. That would be ideal. However, I also think that if it really looks like the general consensus is against something about how glog does things, we shouldn't be married to how glog works. So, overall, I agree with Andrei. The devil is the details, however. - Jonathan M Davis
Re: Implementing std.log
On 05/15/2011 02:54 PM, Jacob Carlborg wrote: On 2011-05-14 19:04, Andrei Alexandrescu wrote: On 5/9/11 1:52 AM, Andrei Alexandrescu wrote: [snip] I updated my std.log draft. Added a lot of features including formatted writing, delayed logging, and a variety of configuration options. Replaced the redundant log.xyz with logXyz. The implementation is getting close to reviewable form. Documentation: http://d-programming-language.org/phobos-prerelease/std_log.html Source: https://github.com/andralex/phobos Feedback welcome. Thanks, Andrei How about an API that looks something like this: http://pastebin.com/dLVp1GRr This API contains standard interfaces for a logger and a logging level. It contains a default logger implementation and a couple of logging levels. This API also allows you to implement custom loggers and custom logging levels. Multiple logger instances are possible with options set on each instance. Note this is just a suggestion for the API and doesn't contain any real implementation (just prints to stdout). Thanks for your work. I think there's an important distinction to be made. There are two APIs being discussed. One is the client interface and the other is the extensibility interface. Jose looked into both: he provided a client interface that has formatting, levels, enabling, and such, and an extensibility interface that essentially is a simple output stream. My library explores the client interface and leaves the extensibility interface as an obvious piece of work that needs little agreement and minimal design effort. Finally, your library keeps the client interface to a minimum and focuses almost exclusively on the extensibility interface. In doing so, it makes few choices that I disagree with. Allow me to share some specific feedback. 1. It becomes dynamic too early. First, in order to disable logging during compilation, a barrier of aliased types is needed. The alias would choose either a null type that does nothing (see StaticNullLogger at https://github.com/andralex/phobos/blob/master/std/log.d), or a log that actually does something. I think your design can be adjusted to do that with relative ease. 2. It becomes dynamic too early from a different perspective. The interface for extensibility is identical with the interface for use and as a consequence leaves too broad functionality unimplemented. A better design is to handle a bunch of decisions (is the log dynamically enabled? every N calls? every N seconds? before N seconds have passed? after N seconds have passed?) in a lightweight front end, i.e. _before_ arguments have been evaluated (this is crucial!). Also, the front end should take care of formatting minutiae, leaving the dynamic back-end the simple task of streaming text where it's supposed to be going. I think Jose's design did the right thing there by doing enabled checks and formatting in the front-end, leaving only transport to the back-end. 3. Another consequence is that extensibility points have the signature: Logger log(string file, long line, ...); which leaves the heavyweight formatting task to the antiquated ... interface (no static types left, meaning e.g. no structs defining toString can be logged). Even assuming that's not a problem, implementors of this function are left with a fair amount of work to do. 4. The Logger class contains info, warning, and error Level objects. Each Level object in turn contains a reference to a Logger. This design leaves some odd shrapnel behind, e.g. one can get to the critical log from the info log. 5. I've seen the notion of a default logging level elsewhere but I think it's inadequate for the info/warning/error/critical/fatal approach. It is adequate for verbosity levels, i.e. vlog and friends. For info/warning/etc. the decision of the appropriate level belongs to the caller. 6. My initial design also contained a global log battery containing the info, warning, etc. objects as members. I found that design stilted because it provides absolutely no benefit besides a marginal log.info vs. logInfo, has no encapsulation advantage, is adverse to extensibility, and worse of all perpetuates poor object oriented design. It's best to call a spade a spade, so there should be five global objects because that's in keep with reality. Andrei
Re: Implementing std.log
On 5/9/11 1:52 AM, Andrei Alexandrescu wrote: [snip] I updated my std.log draft. Added a lot of features including formatted writing, delayed logging, and a variety of configuration options. Replaced the redundant log.xyz with logXyz. The implementation is getting close to reviewable form. Documentation: http://d-programming-language.org/phobos-prerelease/std_log.html Source: https://github.com/andralex/phobos Feedback welcome. Thanks, Andrei
Re: Implementing std.log
On 5/14/2011 1:04 PM, Andrei Alexandrescu wrote: On 5/9/11 1:52 AM, Andrei Alexandrescu wrote: [snip] I updated my std.log draft. Added a lot of features including formatted writing, delayed logging, and a variety of configuration options. Replaced the redundant log.xyz with logXyz. The implementation is getting close to reviewable form. Documentation: http://d-programming-language.org/phobos-prerelease/std_log.html Source: https://github.com/andralex/phobos Feedback welcome. Thanks, Andrei Overall, I like this library a lot. One comment, though, is that the docs should be more explicit about threading issues. I assume it's going to be made thread-safe before inclusion in Phobos. Also, is there any way to get thread-specific logs instead of having them intermingled?
Re: Implementing std.log
On 05/14/2011 01:29 PM, dsimcha wrote: On 5/14/2011 1:04 PM, Andrei Alexandrescu wrote: On 5/9/11 1:52 AM, Andrei Alexandrescu wrote: [snip] I updated my std.log draft. Added a lot of features including formatted writing, delayed logging, and a variety of configuration options. Replaced the redundant log.xyz with logXyz. The implementation is getting close to reviewable form. Documentation: http://d-programming-language.org/phobos-prerelease/std_log.html Source: https://github.com/andralex/phobos Feedback welcome. Thanks, Andrei Overall, I like this library a lot. One comment, though, is that the docs should be more explicit about threading issues. I assume it's going to be made thread-safe before inclusion in Phobos. Also, is there any way to get thread-specific logs instead of having them intermingled? Logs will be thread-shared. I haven't seen a need for thread-local logs. Andrei
Re: Implementing std.log
On 14/05/2011 18:04, Andrei Alexandrescu wrote: On 5/9/11 1:52 AM, Andrei Alexandrescu wrote: [snip] I updated my std.log draft. Added a lot of features including formatted writing, delayed logging, and a variety of configuration options. Replaced the redundant log.xyz with logXyz. The implementation is getting close to reviewable form. Documentation: http://d-programming-language.org/phobos-prerelease/std_log.html Source: https://github.com/andralex/phobos Feedback welcome. Thanks, Andrei I far prefered log.xyz to logXyz... The latter just looks ugly in my opinion. A also notice all those functions return FileLoggers, how will this work when other backends are supported? -- Robert http://octarineparrot.com/
Re: Implementing std.log
On 05/14/2011 02:50 PM, Robert Clipsham wrote: On 14/05/2011 18:04, Andrei Alexandrescu wrote: On 5/9/11 1:52 AM, Andrei Alexandrescu wrote: [snip] I updated my std.log draft. Added a lot of features including formatted writing, delayed logging, and a variety of configuration options. Replaced the redundant log.xyz with logXyz. The implementation is getting close to reviewable form. Documentation: http://d-programming-language.org/phobos-prerelease/std_log.html Source: https://github.com/andralex/phobos Feedback welcome. Thanks, Andrei I far prefered log.xyz to logXyz... The latter just looks ugly in my opinion. There are a few problems with log.xyz. For one, std.log.log.info is terrible. Second, I found absolutely no justification to provide the dummy struct log just to keep five predefined logs in there. Them being individual entities allows the user to define their own logs with the same look and feel, e.g. logRemote. A also notice all those functions return FileLoggers, how will this work when other backends are supported? I want to nail usability before extension. Extensibility can be provided via an indirection inside FileLogger (which should indeed receive a more appropriate name). Andrei
Re: Implementing std.log
On 14/05/2011 21:37, Andrei Alexandrescu wrote: On 05/14/2011 02:50 PM, Robert Clipsham wrote: On 14/05/2011 18:04, Andrei Alexandrescu wrote: On 5/9/11 1:52 AM, Andrei Alexandrescu wrote: [snip] I updated my std.log draft. Added a lot of features including formatted writing, delayed logging, and a variety of configuration options. Replaced the redundant log.xyz with logXyz. The implementation is getting close to reviewable form. Documentation: http://d-programming-language.org/phobos-prerelease/std_log.html Source: https://github.com/andralex/phobos Feedback welcome. Thanks, Andrei I far prefered log.xyz to logXyz... The latter just looks ugly in my opinion. There are a few problems with log.xyz. For one, std.log.log.info is terrible. Second, I found absolutely no justification to provide the dummy struct log just to keep five predefined logs in there. Them being individual entities allows the user to define their own logs with the same look and feel, e.g. logRemote. In which case, wouldn't xyz alone be better? Then you have std.log.info, etc. Renamed imports can be used to get log.info etc. A also notice all those functions return FileLoggers, how will this work when other backends are supported? I want to nail usability before extension. Extensibility can be provided via an indirection inside FileLogger (which should indeed receive a more appropriate name). Fair enough. I keep asking as it's essential functionality for me. -- Robert http://octarineparrot.com/
Re: Implementing std.log
On 2011-05-14 12:36, Andrei Alexandrescu wrote: On 05/14/2011 01:29 PM, dsimcha wrote: On 5/14/2011 1:04 PM, Andrei Alexandrescu wrote: On 5/9/11 1:52 AM, Andrei Alexandrescu wrote: [snip] I updated my std.log draft. Added a lot of features including formatted writing, delayed logging, and a variety of configuration options. Replaced the redundant log.xyz with logXyz. The implementation is getting close to reviewable form. Documentation: http://d-programming-language.org/phobos-prerelease/std_log.html Source: https://github.com/andralex/phobos Feedback welcome. Thanks, Andrei Overall, I like this library a lot. One comment, though, is that the docs should be more explicit about threading issues. I assume it's going to be made thread-safe before inclusion in Phobos. Also, is there any way to get thread-specific logs instead of having them intermingled? Logs will be thread-shared. I haven't seen a need for thread-local logs. I've dealt with code before where it was critical to know which log messages came from which thread if you had any hope of debugging what was going on. However, what I did was have the ID of each thread put at the beginning of every logged line, and it actually would have been a problem for them to be in separate logs, because it would have lost the timing. Of course, part of what I was doing there was tracking down problems with mutexes and race conditions and the like (and the code was _really_ bad about how it handled threads - every thread could be _anywhere_; no segregation at all), and that sort of problem isn't likely to crop up in D code. Still, knowing which thread is logging a particular message can be very useful. However, I question the usefulness of separating the log into thread-specific logs. It might be useful if std.log had an option to print the Thread's name at the beginning of every log line, but if not, someone who wanted that could always just include that at the beginning of their log messages. So, I do think that knowing which thread is logging what could be very important for some programs, but I don't think that separating the log files is necessarily a good idea. If you did, you'd lose timing information (unless the time is at the beginning of every log line (which could also be useful), but then you'd have to read the times and compare them to see what happened before what). So, I'd be all for some options and extra information which could be added to each log line which would help debugging, but I don't think that thread-local logs is a great idea. - Jonathan M Davis
Re: Implementing std.log
On 05/14/2011 04:31 PM, Jonathan M Davis wrote: On 2011-05-14 12:36, Andrei Alexandrescu wrote: On 05/14/2011 01:29 PM, dsimcha wrote: On 5/14/2011 1:04 PM, Andrei Alexandrescu wrote: On 5/9/11 1:52 AM, Andrei Alexandrescu wrote: [snip] I updated my std.log draft. Added a lot of features including formatted writing, delayed logging, and a variety of configuration options. Replaced the redundant log.xyz with logXyz. The implementation is getting close to reviewable form. Documentation: http://d-programming-language.org/phobos-prerelease/std_log.html Source: https://github.com/andralex/phobos Feedback welcome. Thanks, Andrei Overall, I like this library a lot. One comment, though, is that the docs should be more explicit about threading issues. I assume it's going to be made thread-safe before inclusion in Phobos. Also, is there any way to get thread-specific logs instead of having them intermingled? Logs will be thread-shared. I haven't seen a need for thread-local logs. I've dealt with code before where it was critical to know which log messages came from which thread if you had any hope of debugging what was going on. Just like glog, std.log outputs the thread ID for each log line. See the eighth parameter of http://d-programming-language.org/phobos-prerelease/std_log.html#logLinePrefix Andrei
Re: Implementing std.log
On 2011-05-14 17:31:30 -0400, Jonathan M Davis jmdavisp...@gmx.com said: So, I do think that knowing which thread is logging what could be very important for some programs, but I don't think that separating the log files is necessarily a good idea. If you did, you'd lose timing information (unless the time is at the beginning of every log line (which could also be useful), but then you'd have to read the times and compare them to see what happened before what). So, I'd be all for some options and extra information which could be added to each log line which would help debugging, but I don't think that thread-local logs is a great idea. I'd even go further and question whether it makes sense to have info, warning, and errors be written to separate files. I'll also question whether they should be written to files at all by default (as opposed to stdin and stderr). I'm aware initLogging's documentation says: If logÂging is efÂfected withÂout havÂing called this funcÂtion, all paÂraÂmeÂters are at their deÂfault valÂues and all logÂging is done only to stderr. So basically, I'll get what I want if I never call initLogging, but then I can't control verbosity and other settings using --v and other flags passed as arguments. -- Michel Fortin michel.for...@michelf.com http://michelf.com/
Re: Implementing std.log
On 2011-05-14 14:50, Andrei Alexandrescu wrote: On 05/14/2011 04:31 PM, Jonathan M Davis wrote: On 2011-05-14 12:36, Andrei Alexandrescu wrote: On 05/14/2011 01:29 PM, dsimcha wrote: On 5/14/2011 1:04 PM, Andrei Alexandrescu wrote: On 5/9/11 1:52 AM, Andrei Alexandrescu wrote: [snip] I updated my std.log draft. Added a lot of features including formatted writing, delayed logging, and a variety of configuration options. Replaced the redundant log.xyz with logXyz. The implementation is getting close to reviewable form. Documentation: http://d-programming-language.org/phobos-prerelease/std_log.html Source: https://github.com/andralex/phobos Feedback welcome. Thanks, Andrei Overall, I like this library a lot. One comment, though, is that the docs should be more explicit about threading issues. I assume it's going to be made thread-safe before inclusion in Phobos. Also, is there any way to get thread-specific logs instead of having them intermingled? Logs will be thread-shared. I haven't seen a need for thread-local logs. I've dealt with code before where it was critical to know which log messages came from which thread if you had any hope of debugging what was going on. Just like glog, std.log outputs the thread ID for each log line. See the eighth parameter of http://d-programming-language.org/phobos-prerelease/std_log.html#logLinePre fix Ah, good to know. I haven't a had chance to look at it yet, and I'd never even heard of glog before you brought it up on the newsgroup, so I'm not particularly familiar with how the logging is set up. I just thought that I'd share my thoughts on logging separate threads. However, looking at logLinePrefix, it looks quite good. By the way, you should probably add sample strings of what the default format strings generate. Deciphering exactly what a formatted string is going to generate is a lot harder to do than when seeing an example string that was generated from it. - Jonathan M Davis
Re: Implementing std.log
Your post just went asian on me. http://i.imgur.com/solwD.png
Re: Implementing std.log
On 2011-05-14 14:56, Michel Fortin wrote: On 2011-05-14 17:31:30 -0400, Jonathan M Davis jmdavisp...@gmx.com said: So, I do think that knowing which thread is logging what could be very important for some programs, but I don't think that separating the log files is necessarily a good idea. If you did, you'd lose timing information (unless the time is at the beginning of every log line (which could also be useful), but then you'd have to read the times and compare them to see what happened before what). So, I'd be all for some options and extra information which could be added to each log line which would help debugging, but I don't think that thread-local logs is a great idea. I'd even go further and question whether it makes sense to have info, warning, and errors be written to separate files. I'd definitely vote for them all to be in the same file, but I don't generally see much benefit in having multiple log files. I like having them all in one place where you can see what happened in what order. Having them in separate log files is just going to make it harder to figure out what happened, and I think that it would become tempting (for me at least) to just log everything at exactly the same level so that they ended up in the same file. - Jonathan M Davis
Re: Implementing std.log
On 05/14/2011 05:17 PM, Jonathan M Davis wrote: On 2011-05-14 14:56, Michel Fortin wrote: On 2011-05-14 17:31:30 -0400, Jonathan M Davisjmdavisp...@gmx.com said: So, I do think that knowing which thread is logging what could be very important for some programs, but I don't think that separating the log files is necessarily a good idea. If you did, you'd lose timing information (unless the time is at the beginning of every log line (which could also be useful), but then you'd have to read the times and compare them to see what happened before what). So, I'd be all for some options and extra information which could be added to each log line which would help debugging, but I don't think that thread-local logs is a great idea. I'd even go further and question whether it makes sense to have info, warning, and errors be written to separate files. I'd definitely vote for them all to be in the same file, but I don't generally see much benefit in having multiple log files. I like having them all in one place where you can see what happened in what order. Having them in separate log files is just going to make it harder to figure out what happened, and I think that it would become tempting (for me at least) to just log everything at exactly the same level so that they ended up in the same file. The info log contains log messages for all levels. Generally I'd want to stray from glog's major design decision as little as possible. It's an approach validated by years of experience in heavy-duty applications. Also, people coming from glog and used to its features would find it unpleasant that we decided to do things a different way without solid reasons. Andrei
Re: Implementing std.log
On 05/14/2011 04:56 PM, Michel Fortin wrote:
On 2011-05-14 17:31:30 -0400, Jonathan M Davis jmdavisp...@gmx.com said:
So, I do think that knowing which thread is logging what could be very
important for some programs, but I don't think that separating the log
files
is necessarily a good idea. If you did, you'd lose timing information
(unless
the time is at the beginning of every log line (which could also be
useful),
but then you'd have to read the times and compare them to see what
happened
before what). So, I'd be all for some options and extra information which
could be added to each log line which would help debugging, but I
don't think
that thread-local logs is a great idea.
I'd even go further and question whether it makes sense to have info,
warning, and errors be written to separate files.
I'll also question whether they should be written to files at all by
default (as opposed to stdin and stderr). I'm aware initLogging's
documentation says: If logÂging is efÂfected withÂout havÂing called
this funcÂtion, all paÂraÂmeÂters are at their deÂfault valÂues and all
logÂging is done only to stderr. So basically, I'll get what I want if
I never call initLogging, but then I can't control verbosity and other
settings using --v and other flags passed as arguments.
A server app must log to files, no question about that. We'll need to
add rotation etc. in the future. But I do plan to offer ways to manually
override defaults, e.g.:
void main(string[] args)
{
logtostderr = true; // override default
initLogging(args);
...
}
Andrei
Re: Implementing std.log
On 2011-05-14 18:33:06 -0400, Andrei Alexandrescu seewebsiteforem...@erdani.org said: On 05/14/2011 05:17 PM, Jonathan M Davis wrote: On 2011-05-14 14:56, Michel Fortin wrote: I'd even go further and question whether it makes sense to have info, warning, and errors be written to separate files. I'd definitely vote for them all to be in the same file, but I don't generally see much benefit in having multiple log files. I like having them all in one place where you can see what happened in what order. Having them in separate log files is just going to make it harder to figure out what happened, and I think that it would become tempting (for me at least) to just log everything at exactly the same level so that they ended up in the same file. The info log contains log messages for all levels. That's better then, even though it's not very obvious. Perhaps it should be called all instead of info. Is that stated somewhere in the documentation? I seem to have missed it. -- Michel Fortin michel.for...@michelf.com http://michelf.com/
Re: Implementing std.log
On 2011-05-14 13:04:54 -0400, Andrei Alexandrescu seewebsiteforem...@erdani.org said: Documentation: http://d-programming-language.org/phobos-prerelease/std_log.html Source: https://github.com/andralex/phobos Feedback welcome. Shouldn't everyMs and afterMs accept any of core.time's duration types instead of being fixed in milliseconds? -- Michel Fortin michel.for...@michelf.com http://michelf.com/
Re: Implementing std.log
On 2011-05-14 18:35:32 -0400, Andrei Alexandrescu
seewebsiteforem...@erdani.org said:
On 05/14/2011 04:56 PM, Michel Fortin wrote:
I'll also question whether they should be written to files at all by
default (as opposed to stdin and stderr). I'm aware initLogging's
documentation says: If logÂging is efÂfected withÂout havÂing called
this funcÂtion, all paÂraÂmeÂters are at their deÂfault valÂues and all
logÂging is done only to stderr. So basically, I'll get what I want if
I never call initLogging, but then I can't control verbosity and other
settings using --v and other flags passed as arguments.
A server app must log to files, no question about that. We'll need to
add rotation etc. in the future.
No question about that.
What I'm questioning is whether the typical use case for logging will
be a server app. Ideally, I think we should encourage people to add
logs everywhere it makes sense, if only as a debugging aid. If the
default behaviour is to write logs to files it starts to look like a
specialized tools for servers and such and it appears less relevant for
other use cases.
But I do plan to offer ways to manually override defaults, e.g.:
void main(string[] args)
{
logtostderr = true; // override default
initLogging(args);
...
}
The default value for logtostderr should be reversed. Let's not make
a server application the default use case for std.log, shall we? Even
if that was the idea, the default is currently to save files in /tmp
(or equivalent), which does not seem very appropriate for a server to
me.
I think logging to files should be enabled only explicitly by providing
the default logging directory as an argument:
initLogging(args, /logdir);
--
Michel Fortin
michel.for...@michelf.com
http://michelf.com/
Re: Implementing std.log
On 05/14/2011 06:36 PM, Michel Fortin wrote: On 2011-05-14 18:33:06 -0400, Andrei Alexandrescu seewebsiteforem...@erdani.org said: On 05/14/2011 05:17 PM, Jonathan M Davis wrote: On 2011-05-14 14:56, Michel Fortin wrote: I'd even go further and question whether it makes sense to have info, warning, and errors be written to separate files. I'd definitely vote for them all to be in the same file, but I don't generally see much benefit in having multiple log files. I like having them all in one place where you can see what happened in what order. Having them in separate log files is just going to make it harder to figure out what happened, and I think that it would become tempting (for me at least) to just log everything at exactly the same level so that they ended up in the same file. The info log contains log messages for all levels. That's better then, even though it's not very obvious. Perhaps it should be called all instead of info. Is that stated somewhere in the documentation? I seem to have missed it. http://d-programming-language.org/phobos-prerelease/std_log.html Any message logged at a given severity is also logged in logs of lesser severity. Logging to the fatal log always terminates the application after logging. Logging to the critical log always throws an exception after logging. Unless we fix a painful design mistake of glog, I'd keep names as they are. I did diverge from their API with the critical log, which throws. Google doesn't use exceptions so they didn't need a critical log. Andrei
Re: Implementing std.log
On 5/14/2011 3:35 PM, Andrei Alexandrescu wrote: On 05/14/2011 04:56 PM, Michel Fortin wrote: On 2011-05-14 17:31:30 -0400, Jonathan M Davis jmdavisp...@gmx.com said: So, I do think that knowing which thread is logging what could be very important for some programs, but I don't think that separating the log files is necessarily a good idea. If you did, you'd lose timing information (unless the time is at the beginning of every log line (which could also be useful), but then you'd have to read the times and compare them to see what happened before what). So, I'd be all for some options and extra information which could be added to each log line which would help debugging, but I don't think that thread-local logs is a great idea. I'd even go further and question whether it makes sense to have info, warning, and errors be written to separate files. I'll also question whether they should be written to files at all by default (as opposed to stdin and stderr). I'm aware initLogging's documentation says: If logÂging is efÂfected withÂout havÂing called this funcÂtion, all paÂraÂmeÂters are at their deÂfault valÂues and all logÂging is done only to stderr. So basically, I'll get what I want if I never call initLogging, but then I can't control verbosity and other settings using --v and other flags passed as arguments. A server app must log to files, no question about that. We'll need to add rotation etc. in the future. But I do plan to offer ways to manually override defaults, e.g.: Depends on the app. I desire logs for my server to be sent over a socket to a centralized aggregator. But then I've got, um, well, a _lot_ of servers.
Re: Implementing std.log
On 2011-05-14 15:33, Andrei Alexandrescu wrote: On 05/14/2011 05:17 PM, Jonathan M Davis wrote: On 2011-05-14 14:56, Michel Fortin wrote: On 2011-05-14 17:31:30 -0400, Jonathan M Davisjmdavisp...@gmx.com said: So, I do think that knowing which thread is logging what could be very important for some programs, but I don't think that separating the log files is necessarily a good idea. If you did, you'd lose timing information (unless the time is at the beginning of every log line (which could also be useful), but then you'd have to read the times and compare them to see what happened before what). So, I'd be all for some options and extra information which could be added to each log line which would help debugging, but I don't think that thread-local logs is a great idea. I'd even go further and question whether it makes sense to have info, warning, and errors be written to separate files. I'd definitely vote for them all to be in the same file, but I don't generally see much benefit in having multiple log files. I like having them all in one place where you can see what happened in what order. Having them in separate log files is just going to make it harder to figure out what happened, and I think that it would become tempting (for me at least) to just log everything at exactly the same level so that they ended up in the same file. The info log contains log messages for all levels. Generally I'd want to stray from glog's major design decision as little as possible. It's an approach validated by years of experience in heavy-duty applications. Also, people coming from glog and used to its features would find it unpleasant that we decided to do things a different way without solid reasons. If it's based on glog, it makes sense for it to generally stick to what glog does. However, if it's forcing you to put all of the different log levels in different files, then I'd argue that that was a poor design choice on the part of glog and that we should not stick to it. However, if they all end up in the info log, then it's not that bad. Now, I'd still love to be able to tell it only have the info log then and not bother with multiple logs, but at least it's possible to get it all in one log. - Jonathan M Davis
Re: Implementing std.log
On 2011-05-14 16:35, Michel Fortin wrote: On 2011-05-14 13:04:54 -0400, Andrei Alexandrescu seewebsiteforem...@erdani.org said: Documentation: http://d-programming-language.org/phobos-prerelease/std_log.html Source: https://github.com/andralex/phobos Feedback welcome. Shouldn't everyMs and afterMs accept any of core.time's duration types instead of being fixed in milliseconds? I'd vote for that. Then again, I'm tempted to argue that we shouldn't have _any_ time-related functions which take naked numbers. There are several time- related functions in druntime and Phobos which still have versions which take naked numbers. They may all now have versions which take a core.time.Duration, but most of them still have versions which takes a naked number and which have not be scheduled for deprecation. Now, there may be a valid argument for keeping versions of some functions around which take a naked number instead of a Duration, but I really think that we should lean towards using Durationl for that sort of thing and get rid of those that take naked numbers. It's less error-prone and would increase the consistency of how times are handled in druntime and Phobos. - Jonathan M Davis
Re: Implementing std.log
On 5/14/11 6:35 PM, Michel Fortin wrote: On 2011-05-14 13:04:54 -0400, Andrei Alexandrescu seewebsiteforem...@erdani.org said: Documentation: http://d-programming-language.org/phobos-prerelease/std_log.html Source: https://github.com/andralex/phobos Feedback welcome. Shouldn't everyMs and afterMs accept any of core.time's duration types instead of being fixed in milliseconds? Great idea. Then we can actually overload every() and after(). Thanks, Andrei
Re: Implementing std.log
On 5/14/11 6:43 PM, Michel Fortin wrote: On 2011-05-14 18:35:32 -0400, Andrei Alexandrescu seewebsiteforem...@erdani.org said: On 05/14/2011 04:56 PM, Michel Fortin wrote: I'll also question whether they should be written to files at all by default (as opposed to stdin and stderr). I'm aware initLogging's documentation says: If logÂging is efÂfected withÂout havÂing called this funcÂtion, all paÂraÂmeÂters are at their deÂfault valÂues and all logÂging is done only to stderr. So basically, I'll get what I want if I never call initLogging, but then I can't control verbosity and other settings using --v and other flags passed as arguments. A server app must log to files, no question about that. We'll need to add rotation etc. in the future. No question about that. What I'm questioning is whether the typical use case for logging will be a server app. Ideally, I think we should encourage people to add logs everywhere it makes sense, if only as a debugging aid. If the default behaviour is to write logs to files it starts to look like a specialized tools for servers and such and it appears less relevant for other use cases. In fact it may as well be the case that logging is for servers first. For casual use there's always writeln. Andrei
Re: Implementing std.log
On 5/14/11 7:13 PM, Brad Roberts wrote: On 5/14/2011 3:35 PM, Andrei Alexandrescu wrote: On 05/14/2011 04:56 PM, Michel Fortin wrote: On 2011-05-14 17:31:30 -0400, Jonathan M Davisjmdavisp...@gmx.com said: So, I do think that knowing which thread is logging what could be very important for some programs, but I don't think that separating the log files is necessarily a good idea. If you did, you'd lose timing information (unless the time is at the beginning of every log line (which could also be useful), but then you'd have to read the times and compare them to see what happened before what). So, I'd be all for some options and extra information which could be added to each log line which would help debugging, but I don't think that thread-local logs is a great idea. I'd even go further and question whether it makes sense to have info, warning, and errors be written to separate files. I'll also question whether they should be written to files at all by default (as opposed to stdin and stderr). I'm aware initLogging's documentation says: If logÂging is efÂfected withÂout havÂing called this funcÂtion, all paÂraÂmeÂters are at their deÂfault valÂues and all logÂging is done only to stderr. So basically, I'll get what I want if I never call initLogging, but then I can't control verbosity and other settings using --v and other flags passed as arguments. A server app must log to files, no question about that. We'll need to add rotation etc. in the future. But I do plan to offer ways to manually override defaults, e.g.: Depends on the app. I desire logs for my server to be sent over a socket to a centralized aggregator. But then I've got, um, well, a _lot_ of servers. We'll need to provide socket streaming support. Andrei
Re: Implementing std.log
On 2011-05-14 10:04, Andrei Alexandrescu wrote: On 5/9/11 1:52 AM, Andrei Alexandrescu wrote: [snip] I updated my std.log draft. Added a lot of features including formatted writing, delayed logging, and a variety of configuration options. Replaced the redundant log.xyz with logXyz. The implementation is getting close to reviewable form. Documentation: http://d-programming-language.org/phobos-prerelease/std_log.html Source: https://github.com/andralex/phobos Feedback welcome. A minor note on efficiency. You're currently calling hour, minute, second, etc. on the result of Clock.currTime() (which is a SysTime) and doing so in a loop. Every one of those calls has to convert the internal representation in hnsecs to the units your asking for. It would be more efficient to take the result of Clock.currTime() and then save a DateTime and FracSec with the values from that SysTime. For example, instead of auto t = Clock.currTime(); ... formattedWrite(writer, prefix, level, t.month, t.day, t.hour, t.minute, t.second, t.fracSec.usecs, tid, fdir, fname, n); you could do auto t = Clock.currTime(); auto dt = cast(DateTime)t; auto fs = t.fracSec; ... formattedWrite(writer, prefix, level, dt.month, dt.day, dt.hour, dt.minute, dt.second, fs.fracSec.usecs, tid, fdir, fname, n); It's not a big deal. SysTime has all of the functions on it, and it works the way that you're doing it. It's just less efficienty since it has to do more calculations, whereas if you get a DateTime and FracSec from it, there are far fewer calcuations to do. Actually, I should probably add a note about that to SysTime's documentation. It should be fairly obvious if you sit down and think about it, but obviously people aren't usually going to be doing that, so it's likely unreasonable to expect people to figure that out on their own short actually needing to find ways to improve the efficiency of their code date- time. In any case, I just thought that I'd point that out. I'm not sure if it will really have much of an effect (especially when I/O is going to be the bottlneck for logging, not stray date-time calculations), but logging does need to be efficient. - Jonathan M Davis
Re: Implementing std.log
On 2011-05-12 05:38, Andrei Alexandrescu wrote: On 5/11/11 5:09 PM, Sean Kelly wrote: On May 11, 2011, at 11:38 AM, Jacob Carlborg wrote: On 2011-05-10 16:41, Andrei Alexandrescu wrote: On 5/10/11 3:22 AM, Jacob Carlborg wrote: Are you referring to core.runtime.Runtime.args? That is not completely reliable because: * You can start a new process, with exec, and then pass in whatever you want as the first argument to the process. * If you start an application via a symlink wouldn't that refer to the symlink instead of the actual executable? * Also if you're running an application in a bundle on Mac OS X it would refer to the bundle and not the actual executable, if I recall correctly. I see. I'll defer ultimate decision on adding getProcessPath to Sean and Walter. Probably with a little experimentation to clarify motivation, getProcessPath is a worthy addition. Sean, Walter, is this worth including? Yes. Not sure where it should live though. http://d-programming-language.org/phobos/std_process.html Jacob, please write a pull request with documentation (including motivation by comparison with core.Runtime.args), unittests, the works, and provided the quality is up to snuff we'll make sure we merge it in. Thanks! Andrei I'll do this as soon as I can find the time, although it's not on the top of my priority list. -- /Jacob Carlborg
Re: Implementing std.log
Andrei Alexandrescu wrote: On 5/9/11 3:12 PM, Jens Mueller wrote: I think every() behaves strangely. Because the counter is per function. But it should be per logging statement. std.log differs here from glog and this seems incorrect to me. everyMs() has a similar problem. Actually, they behave correctly. The counters for every and everyMs are static and generated per instantiation of the function. Since each instantiation is determined by __FILE__ and __LINE__, there is a different counters for each __FILE__/__LINE__ combination. Oh yes. Now I see. I missed the template arguments. Sorry. Now I also understand why you want to cut down the possible code bloat. Even though I believe log.vlog is necessary I often find it difficult to come up with an appropriate level for a given statement. With normal logging it is often obvious whether info, warning, error, critical, or fatal is appropriate. But this is a minor problem and more a matter of coding style. No need to use vlog. But a variety of programs (e.g. ssh) decide quite precisely what will be logged at each verbosity level. Maybe it helps that you provide -v as incremental options. Glog does not do it like this. But I think allowing incremental options helps here. It tells the user to use vlog(0) for the least verbose messages and so on incrementally for more verbosity. I'd like to try out the code. I'd be very pleased if you can make a compiling version available on github. I'd like to test whether there is no code generated if -version=strip_log_error etc. is specified. I.e. whether the call to an empty function will always be removed. Thanks for your interest. I just committed the code to my repo: https://github.com/andralex/phobos You'd need the latest dmd and druntime to compile Phobos. Thanks. I'll try it out. Jens
Re: Implementing std.log
On 5/9/2011 2:52 AM, Andrei Alexandrescu wrote: As I mentioned I'd already decided I'll take a stab at implementing a logging module inspired in design from glog. I was tired of the endless discussions on what a logging API should look like. This ironically is leading now to an embarrassment of riches - we now have two proposals on the table. I subjectively prefer mine for the simple reason that it includes exactly what I wanted from a logging subsystem with a light syntax. I noticed this was recently checked into Phobos. What's its status? Is it experimental? Is it for inclusion in the upcoming (2.053) release? Is it for 2.054 (the next release after)?
Re: Implementing std.log
On 05/11/2011 08:46 AM, dsimcha wrote: On 5/9/2011 2:52 AM, Andrei Alexandrescu wrote: As I mentioned I'd already decided I'll take a stab at implementing a logging module inspired in design from glog. I was tired of the endless discussions on what a logging API should look like. This ironically is leading now to an embarrassment of riches - we now have two proposals on the table. I subjectively prefer mine for the simple reason that it includes exactly what I wanted from a logging subsystem with a light syntax. I noticed this was recently checked into Phobos. What's its status? Is it experimental? Is it for inclusion in the upcoming (2.053) release? Is it for 2.054 (the next release after)? Ouch. Pushing to Phobos is a mistake, I meant to only push it to my own fork of Phobos. I wouldn't dream of pushing a new module without going through the review process, and even less so since there's a competing proposal. How do I undo that push? Thanks, Andrei
Re: Implementing std.log
On 05/11/2011 09:15 AM, Andrei Alexandrescu wrote: On 05/11/2011 08:46 AM, dsimcha wrote: On 5/9/2011 2:52 AM, Andrei Alexandrescu wrote: As I mentioned I'd already decided I'll take a stab at implementing a logging module inspired in design from glog. I was tired of the endless discussions on what a logging API should look like. This ironically is leading now to an embarrassment of riches - we now have two proposals on the table. I subjectively prefer mine for the simple reason that it includes exactly what I wanted from a logging subsystem with a light syntax. I noticed this was recently checked into Phobos. What's its status? Is it experimental? Is it for inclusion in the upcoming (2.053) release? Is it for 2.054 (the next release after)? Ouch. Pushing to Phobos is a mistake, I meant to only push it to my own fork of Phobos. I wouldn't dream of pushing a new module without going through the review process, and even less so since there's a competing proposal. How do I undo that push? Thanks, Andrei I removed std.log from Phobos. I left in support for positional parameter ranges, e.g. writefln(%1$u:%2:3$s-%4:$s, ...); prints the first argument with spec u, the second and third with the spec s, and everything else with the spec s. Andrei
Re: Implementing std.log
On 2011-05-10 16:41, Andrei Alexandrescu wrote: On 5/10/11 3:22 AM, Jacob Carlborg wrote: Are you referring to core.runtime.Runtime.args? That is not completely reliable because: * You can start a new process, with exec, and then pass in whatever you want as the first argument to the process. * If you start an application via a symlink wouldn't that refer to the symlink instead of the actual executable? * Also if you're running an application in a bundle on Mac OS X it would refer to the bundle and not the actual executable, if I recall correctly. I see. I'll defer ultimate decision on adding getProcessPath to Sean and Walter. Probably with a little experimentation to clarify motivation, getProcessPath is a worthy addition. Thanks, Andrei Sean, Walter, is this worth including? -- /Jacob Carlborg
Re: Implementing std.log
On May 11, 2011, at 11:38 AM, Jacob Carlborg wrote: On 2011-05-10 16:41, Andrei Alexandrescu wrote: On 5/10/11 3:22 AM, Jacob Carlborg wrote: Are you referring to core.runtime.Runtime.args? That is not completely reliable because: * You can start a new process, with exec, and then pass in whatever you want as the first argument to the process. * If you start an application via a symlink wouldn't that refer to the symlink instead of the actual executable? * Also if you're running an application in a bundle on Mac OS X it would refer to the bundle and not the actual executable, if I recall correctly. I see. I'll defer ultimate decision on adding getProcessPath to Sean and Walter. Probably with a little experimentation to clarify motivation, getProcessPath is a worthy addition. Sean, Walter, is this worth including? Yes. Not sure where it should live though.
Re: Implementing std.log
On 5/11/11 5:09 PM, Sean Kelly wrote: On May 11, 2011, at 11:38 AM, Jacob Carlborg wrote: On 2011-05-10 16:41, Andrei Alexandrescu wrote: On 5/10/11 3:22 AM, Jacob Carlborg wrote: Are you referring to core.runtime.Runtime.args? That is not completely reliable because: * You can start a new process, with exec, and then pass in whatever you want as the first argument to the process. * If you start an application via a symlink wouldn't that refer to the symlink instead of the actual executable? * Also if you're running an application in a bundle on Mac OS X it would refer to the bundle and not the actual executable, if I recall correctly. I see. I'll defer ultimate decision on adding getProcessPath to Sean and Walter. Probably with a little experimentation to clarify motivation, getProcessPath is a worthy addition. Sean, Walter, is this worth including? Yes. Not sure where it should live though. http://d-programming-language.org/phobos/std_process.html Jacob, please write a pull request with documentation (including motivation by comparison with core.Runtime.args), unittests, the works, and provided the quality is up to snuff we'll make sure we merge it in. Thanks! Andrei
Re: Implementing std.log
On 2011-05-09 21:18, Andrei Alexandrescu wrote:
On 5/9/11 1:48 PM, Jacob Carlborg wrote:
On 2011-05-09 19:58, Andrei Alexandrescu wrote:
On 5/9/11 12:24 PM, Jacob Carlborg wrote:
I
have a function that gets the path of the current process:
http://dsource.org/projects/tango/attachment/ticket/1536/process.d
This links to a file attached to a Tango ticket but don't worry, I've
written the whole file myself and I intend to change the license to the
Boost License. It's written with D1 (obviously) but it's very easy to
port to D2, the only dependencies are C functions.
Feel free to port it to D2 and use it in you're logging library if you
want to.
I looked over the code, it's quite nice and clean. If you or anyone else
would want to create a github pull request, the function would be a
valuable addition to std.process. Thanks Jacob for offering the code to
Phobos!
What I think should change:
* Provide two overloads, one that attempts to reuse a buffer and one
that always returns a new string:
char[] getProcessPath(char[] buf) { ... }
static if (is(typeof(getProcessPath(null)) == char[]))
{
string getProcessPath() { return assumeUnique(getProcessPath(null)); }
}
Not sure I understand this.
The static if actually refers to the next point, which implies that
getProcessPath(char[]) may not exist on some systems.
The idea behind having two functions is to give a string to people who
just want a string without messing with buffers and all. The string has
the nice property that nobody can trample on it later.
Ok, I'll see if I can find the time to create a pull request out of
this. In what module should I put the function?
It looks like core.runtime already allows fetching the process path
name, so you need to figure out what the added value of your function is.
Andrei
Are you referring to core.runtime.Runtime.args? That is not completely
reliable because:
* You can start a new process, with exec, and then pass in whatever you
want as the first argument to the process.
* If you start an application via a symlink wouldn't that refer to the
symlink instead of the actual executable?
* Also if you're running an application in a bundle on Mac OS X it would
refer to the bundle and not the actual executable, if I recall correctly.
--
/Jacob Carlborg
Re: Implementing std.log
On 5/10/11 3:22 AM, Jacob Carlborg wrote:
On 2011-05-09 21:18, Andrei Alexandrescu wrote:
On 5/9/11 1:48 PM, Jacob Carlborg wrote:
On 2011-05-09 19:58, Andrei Alexandrescu wrote:
On 5/9/11 12:24 PM, Jacob Carlborg wrote:
I
have a function that gets the path of the current process:
http://dsource.org/projects/tango/attachment/ticket/1536/process.d
This links to a file attached to a Tango ticket but don't worry, I've
written the whole file myself and I intend to change the license to
the
Boost License. It's written with D1 (obviously) but it's very easy to
port to D2, the only dependencies are C functions.
Feel free to port it to D2 and use it in you're logging library if you
want to.
I looked over the code, it's quite nice and clean. If you or anyone
else
would want to create a github pull request, the function would be a
valuable addition to std.process. Thanks Jacob for offering the code to
Phobos!
What I think should change:
* Provide two overloads, one that attempts to reuse a buffer and one
that always returns a new string:
char[] getProcessPath(char[] buf) { ... }
static if (is(typeof(getProcessPath(null)) == char[]))
{
string getProcessPath() { return assumeUnique(getProcessPath(null)); }
}
Not sure I understand this.
The static if actually refers to the next point, which implies that
getProcessPath(char[]) may not exist on some systems.
The idea behind having two functions is to give a string to people who
just want a string without messing with buffers and all. The string has
the nice property that nobody can trample on it later.
Ok, I'll see if I can find the time to create a pull request out of
this. In what module should I put the function?
It looks like core.runtime already allows fetching the process path
name, so you need to figure out what the added value of your function is.
Andrei
Are you referring to core.runtime.Runtime.args? That is not completely
reliable because:
* You can start a new process, with exec, and then pass in whatever you
want as the first argument to the process.
* If you start an application via a symlink wouldn't that refer to the
symlink instead of the actual executable?
* Also if you're running an application in a bundle on Mac OS X it would
refer to the bundle and not the actual executable, if I recall correctly.
I see. I'll defer ultimate decision on adding getProcessPath to Sean and
Walter. Probably with a little experimentation to clarify motivation,
getProcessPath is a worthy addition.
Thanks,
Andrei
Re: Implementing std.log
On 2011-05-10 16:41, Andrei Alexandrescu wrote:
On 5/10/11 3:22 AM, Jacob Carlborg wrote:
On 2011-05-09 21:18, Andrei Alexandrescu wrote:
On 5/9/11 1:48 PM, Jacob Carlborg wrote:
On 2011-05-09 19:58, Andrei Alexandrescu wrote:
On 5/9/11 12:24 PM, Jacob Carlborg wrote:
I
have a function that gets the path of the current process:
http://dsource.org/projects/tango/attachment/ticket/1536/process.d
This links to a file attached to a Tango ticket but don't worry, I've
written the whole file myself and I intend to change the license to
the
Boost License. It's written with D1 (obviously) but it's very easy to
port to D2, the only dependencies are C functions.
Feel free to port it to D2 and use it in you're logging library if
you
want to.
I looked over the code, it's quite nice and clean. If you or anyone
else
would want to create a github pull request, the function would be a
valuable addition to std.process. Thanks Jacob for offering the
code to
Phobos!
What I think should change:
* Provide two overloads, one that attempts to reuse a buffer and one
that always returns a new string:
char[] getProcessPath(char[] buf) { ... }
static if (is(typeof(getProcessPath(null)) == char[]))
{
string getProcessPath() { return assumeUnique(getProcessPath(null)); }
}
Not sure I understand this.
The static if actually refers to the next point, which implies that
getProcessPath(char[]) may not exist on some systems.
The idea behind having two functions is to give a string to people who
just want a string without messing with buffers and all. The string has
the nice property that nobody can trample on it later.
Ok, I'll see if I can find the time to create a pull request out of
this. In what module should I put the function?
It looks like core.runtime already allows fetching the process path
name, so you need to figure out what the added value of your function
is.
Andrei
Are you referring to core.runtime.Runtime.args? That is not completely
reliable because:
* You can start a new process, with exec, and then pass in whatever you
want as the first argument to the process.
* If you start an application via a symlink wouldn't that refer to the
symlink instead of the actual executable?
* Also if you're running an application in a bundle on Mac OS X it would
refer to the bundle and not the actual executable, if I recall correctly.
I see. I'll defer ultimate decision on adding getProcessPath to Sean and
Walter. Probably with a little experimentation to clarify motivation,
getProcessPath is a worthy addition.
Thanks,
Andrei
Well, I only brought it up because I didn't know about Runtime.args.
--
/Jacob Carlborg
Re: Implementing std.log
Thanks! Will do. On Mon, May 9, 2011 at 4:11 PM, Walter Bright newshou...@digitalmars.com wrote: On 5/7/2011 1:43 PM, Jose Armando Garcia wrote: My intent, and hopefully we will get there with your help, is to include this in Phobos for D2. Thanks for doing the hard work of designing and laying out an implementation. Please, though, posting the source code as a news group message is not very efficient. Much better is to create an account with github, and post it there. Github has a lot of nice features that make collaboration, commenting, and incorporation easy, while a n.g. posting has none of that.
Re: Implementing std.log
On 5/9/11 3:12 PM, Jens Mueller wrote: I think every() behaves strangely. Because the counter is per function. But it should be per logging statement. std.log differs here from glog and this seems incorrect to me. everyMs() has a similar problem. Actually, they behave correctly. The counters for every and everyMs are static and generated per instantiation of the function. Since each instantiation is determined by __FILE__ and __LINE__, there is a different counters for each __FILE__/__LINE__ combination. Even though I believe log.vlog is necessary I often find it difficult to come up with an appropriate level for a given statement. With normal logging it is often obvious whether info, warning, error, critical, or fatal is appropriate. But this is a minor problem and more a matter of coding style. No need to use vlog. But a variety of programs (e.g. ssh) decide quite precisely what will be logged at each verbosity level. I'd like to try out the code. I'd be very pleased if you can make a compiling version available on github. I'd like to test whether there is no code generated if -version=strip_log_error etc. is specified. I.e. whether the call to an empty function will always be removed. Thanks for your interest. I just committed the code to my repo: https://github.com/andralex/phobos You'd need the latest dmd and druntime to compile Phobos. Andrei
Re: Implementing std.log
On 5/7/11 3:43 PM, Jose Armando Garcia wrote: Hey folks, For the past couple of days I took the liberty of partially implementing a logging module for D. [snip] Nice work. Also thanks for contacting me offline. As I mentioned I'd already decided I'll take a stab at implementing a logging module inspired in design from glog. I was tired of the endless discussions on what a logging API should look like. This ironically is leading now to an embarrassment of riches - we now have two proposals on the table. I subjectively prefer mine for the simple reason that it includes exactly what I wanted from a logging subsystem with a light syntax. The documentation is at http://d-programming-language.org/phobos-prerelease/std_log.html. The source code is at http://erdani.com/log.d, with informative title only; to compile the code you'd need some changes in std.format as well (I extended the positional parameter syntax to allow ranges of positions). Todos include slight refactoring to avoid bloat in generated code per call, OS portability (I only tested on OSX), and getting threading right by fixing the shared-related compiler bugs. Looking forward to more discussion of the matter. Andrei
Re: Implementing std.log
On 2011-05-09 08:52, Andrei Alexandrescu wrote: On 5/7/11 3:43 PM, Jose Armando Garcia wrote: Hey folks, For the past couple of days I took the liberty of partially implementing a logging module for D. [snip] Nice work. Also thanks for contacting me offline. As I mentioned I'd already decided I'll take a stab at implementing a logging module inspired in design from glog. I was tired of the endless discussions on what a logging API should look like. This ironically is leading now to an embarrassment of riches - we now have two proposals on the table. I subjectively prefer mine for the simple reason that it includes exactly what I wanted from a logging subsystem with a light syntax. The documentation is at http://d-programming-language.org/phobos-prerelease/std_log.html. The source code is at http://erdani.com/log.d, with informative title only; to compile the code you'd need some changes in std.format as well (I extended the positional parameter syntax to allow ranges of positions). Todos include slight refactoring to avoid bloat in generated code per call, OS portability (I only tested on OSX), and getting threading right by fixing the shared-related compiler bugs. Looking forward to more discussion of the matter. Andrei Isn't the name of the log file a little too verbose? Don't know if I'm missing something but this doesn't seem so useful for long running application that use logging. As far as I know you usually have one log file per application and outputs the message along with the level and date and time. -- /Jacob Carlborg
Re: Implementing std.log
On 5/9/11 10:37 AM, Jacob Carlborg wrote: Isn't the name of the log file a little too verbose? Don't know if I'm missing something but this doesn't seem so useful for long running application that use logging. As far as I know you usually have one log file per application and outputs the message along with the level and date and time. I did what glog does. Going forward, there is a private static string for formatting file names using positional parameters. In the future I'm considering providing an API for changing that format string, which means the log file format will be user-definable. (Same about the format of individual messages - see the string constants fileNameFormat and format toward the top of log.d.) Positional parameters look a bit odd but they are quite flexible. Andrei
Re: Implementing std.log
On 2011-05-09 17:49, Andrei Alexandrescu wrote: On 5/9/11 10:37 AM, Jacob Carlborg wrote: Isn't the name of the log file a little too verbose? Don't know if I'm missing something but this doesn't seem so useful for long running application that use logging. As far as I know you usually have one log file per application and outputs the message along with the level and date and time. I did what glog does. Going forward, there is a private static string for formatting file names using positional parameters. In the future I'm considering providing an API for changing that format string, which means the log file format will be user-definable. (Same about the format of individual messages - see the string constants fileNameFormat and format toward the top of log.d.) Positional parameters look a bit odd but they are quite flexible. Andrei Excuse me for asking again but to me it seems that the log functions will output a new file for every log event (since the time is included in the filename). Is that the case? -- /Jacob Carlborg
Re: Implementing std.log
On 2011-05-09 17:49, Andrei Alexandrescu wrote: On 5/9/11 10:37 AM, Jacob Carlborg wrote: Isn't the name of the log file a little too verbose? Don't know if I'm missing something but this doesn't seem so useful for long running application that use logging. As far as I know you usually have one log file per application and outputs the message along with the level and date and time. I did what glog does. Going forward, there is a private static string for formatting file names using positional parameters. In the future I'm considering providing an API for changing that format string, which means the log file format will be user-definable. (Same about the format of individual messages - see the string constants fileNameFormat and format toward the top of log.d.) Positional parameters look a bit odd but they are quite flexible. Andrei I don't think the positional parameters are odd (C# has them with another syntax and Tango uses the same syntax as C#), I think they're very usable. Although the syntax you've chosen looks a little odd to me. Wouldn't it make more sense to have the $ in front of the number instead of behind, since the % is in front. -- /Jacob Carlborg
Re: Implementing std.log
On 2011-05-09 08:52, Andrei Alexandrescu wrote: On 5/7/11 3:43 PM, Jose Armando Garcia wrote: Hey folks, For the past couple of days I took the liberty of partially implementing a logging module for D. [snip] Nice work. Also thanks for contacting me offline. As I mentioned I'd already decided I'll take a stab at implementing a logging module inspired in design from glog. I was tired of the endless discussions on what a logging API should look like. This ironically is leading now to an embarrassment of riches - we now have two proposals on the table. I subjectively prefer mine for the simple reason that it includes exactly what I wanted from a logging subsystem with a light syntax. The documentation is at http://d-programming-language.org/phobos-prerelease/std_log.html. The source code is at http://erdani.com/log.d, with informative title only; to compile the code you'd need some changes in std.format as well (I extended the positional parameter syntax to allow ranges of positions). Todos include slight refactoring to avoid bloat in generated code per call, OS portability (I only tested on OSX), and getting threading right by fixing the shared-related compiler bugs. Looking forward to more discussion of the matter. Andrei I assume the initLogging function needs the application command line because you want to have the name of the application? If that's the case, I don't like that is handled via the application command line. I have a function that gets the path of the current process: http://dsource.org/projects/tango/attachment/ticket/1536/process.d This links to a file attached to a Tango ticket but don't worry, I've written the whole file myself and I intend to change the license to the Boost License. It's written with D1 (obviously) but it's very easy to port to D2, the only dependencies are C functions. Feel free to port it to D2 and use it in you're logging library if you want to. -- /Jacob Carlborg
Re: Implementing std.log
On 5/9/11 12:10 PM, Jacob Carlborg wrote: On 2011-05-09 17:49, Andrei Alexandrescu wrote: On 5/9/11 10:37 AM, Jacob Carlborg wrote: Isn't the name of the log file a little too verbose? Don't know if I'm missing something but this doesn't seem so useful for long running application that use logging. As far as I know you usually have one log file per application and outputs the message along with the level and date and time. I did what glog does. Going forward, there is a private static string for formatting file names using positional parameters. In the future I'm considering providing an API for changing that format string, which means the log file format will be user-definable. (Same about the format of individual messages - see the string constants fileNameFormat and format toward the top of log.d.) Positional parameters look a bit odd but they are quite flexible. Andrei Excuse me for asking again but to me it seems that the log functions will output a new file for every log event (since the time is included in the filename). Is that the case? No, log files are created and opened only once, and the time in the filename reflects the creation time. Logging messages also contain the time (with added precision) and go to the respective files. A maximum of five files are created. (No support for rotating logs for the time being.) There's one difference from glog by the way - glog defines four levels and consequently four files: info, warning, error, and fatal. I defined critical in between error and fatal. Logging to the critical log throws an exception right after logging. I think this is very useful for figuring out what happens in exceptional code. Andrei
Re: Implementing std.log
On 5/9/11 12:14 PM, Jacob Carlborg wrote: On 2011-05-09 17:49, Andrei Alexandrescu wrote: On 5/9/11 10:37 AM, Jacob Carlborg wrote: Isn't the name of the log file a little too verbose? Don't know if I'm missing something but this doesn't seem so useful for long running application that use logging. As far as I know you usually have one log file per application and outputs the message along with the level and date and time. I did what glog does. Going forward, there is a private static string for formatting file names using positional parameters. In the future I'm considering providing an API for changing that format string, which means the log file format will be user-definable. (Same about the format of individual messages - see the string constants fileNameFormat and format toward the top of log.d.) Positional parameters look a bit odd but they are quite flexible. Andrei I don't think the positional parameters are odd (C# has them with another syntax and Tango uses the same syntax as C#), I think they're very usable. Although the syntax you've chosen looks a little odd to me. Wouldn't it make more sense to have the $ in front of the number instead of behind, since the % is in front. I used the Posix positional format syntax, see http://pubs.opengroup.org/onlinepubs/009695399/functions/printf.html. Andrei
Re: Implementing std.log
On 5/9/11 12:24 PM, Jacob Carlborg wrote: I assume the initLogging function needs the application command line because you want to have the name of the application? If that's the case, I don't like that is handled via the application command line. I have a function that gets the path of the current process: http://dsource.org/projects/tango/attachment/ticket/1536/process.d This links to a file attached to a Tango ticket but don't worry, I've written the whole file myself and I intend to change the license to the Boost License. It's written with D1 (obviously) but it's very easy to port to D2, the only dependencies are C functions. Feel free to port it to D2 and use it in you're logging library if you want to. Thanks! In fact, the command-line arguments are needed for considerably more than just the application name. std.log is configurable via application's command line (e.g. --logtostderr --log_dir=/tmp/ etc., see documentation) and extracts these flags using getopt. The rest of the flags are left alone for the application to process. In case the application would want to change a flag's name, e.g. use --log_to_stderr instead of --logtostderr, it's easy to define an API that does that; currently all parameter names are hoisted in private strings in std.log. Andrei
Re: Implementing std.log
On 5/9/11 12:24 PM, Jacob Carlborg wrote:
I
have a function that gets the path of the current process:
http://dsource.org/projects/tango/attachment/ticket/1536/process.d
This links to a file attached to a Tango ticket but don't worry, I've
written the whole file myself and I intend to change the license to the
Boost License. It's written with D1 (obviously) but it's very easy to
port to D2, the only dependencies are C functions.
Feel free to port it to D2 and use it in you're logging library if you
want to.
I looked over the code, it's quite nice and clean. If you or anyone else
would want to create a github pull request, the function would be a
valuable addition to std.process. Thanks Jacob for offering the code to
Phobos!
What I think should change:
* Provide two overloads, one that attempts to reuse a buffer and one
that always returns a new string:
char[] getProcessPath(char[] buf) { ... }
static if (is(typeof(getProcessPath(null)) == char[]))
{
string getProcessPath() { return assumeUnique(getProcessPath(null)); }
}
* hoist version(OS) up to the function definition, e.g.
version (darwin) char[] getProcessPath (char[] buf) { ... }
version (freebsd) char[] getProcessPath (char[] buf) { ... }
...
Unsupported OSs will simply not define the function (as opposed to
asserting at run time).
* Use size_t instead of uint throughout for 64-bit compatibility.
* Don't do this:
if (size buf.length)
buf ~= new char[size - buf.length];
because it allocates memory twice. Instead:
if (size buf.length)
buf.length = size;
which will allocate memory once or not at all.
* Check for all system calls for errors.
Again, thanks Jacob for your contribution!
Andrei
Re: Implementing std.log
On May 9, 2011, at 10:45 AM, Andrei Alexandrescu wrote: On 5/9/11 12:24 PM, Jacob Carlborg wrote: I assume the initLogging function needs the application command line because you want to have the name of the application? If that's the case, I don't like that is handled via the application command line. I have a function that gets the path of the current process: http://dsource.org/projects/tango/attachment/ticket/1536/process.d This links to a file attached to a Tango ticket but don't worry, I've written the whole file myself and I intend to change the license to the Boost License. It's written with D1 (obviously) but it's very easy to port to D2, the only dependencies are C functions. Feel free to port it to D2 and use it in you're logging library if you want to. Thanks! In fact, the command-line arguments are needed for considerably more than just the application name. std.log is configurable via application's command line (e.g. --logtostderr --log_dir=/tmp/ etc., see documentation) and extracts these flags using getopt. The rest of the flags are left alone for the application to process. For what it's worth, the command-line args are available via core.runtime as well.
Re: Implementing std.log
I'm not a big fan of that format syntax. It's just hard to look at and
figure out what went wrong when things do go wrong.
I'd prefer something like:
https://gist.github.com/963027
But someone would have to implement it, of course. I have my own
little echo() function which takes simple {} positional syntax, it
doesn't have any other formatting options though. But I only ever need
formatting for simple cases like the above. It seems some people need
a whole word processor macro language embedded in a format string..
Re: Implementing std.log
On 2011-05-09 19:34, Andrei Alexandrescu wrote: On 5/9/11 12:14 PM, Jacob Carlborg wrote: On 2011-05-09 17:49, Andrei Alexandrescu wrote: On 5/9/11 10:37 AM, Jacob Carlborg wrote: Isn't the name of the log file a little too verbose? Don't know if I'm missing something but this doesn't seem so useful for long running application that use logging. As far as I know you usually have one log file per application and outputs the message along with the level and date and time. I did what glog does. Going forward, there is a private static string for formatting file names using positional parameters. In the future I'm considering providing an API for changing that format string, which means the log file format will be user-definable. (Same about the format of individual messages - see the string constants fileNameFormat and format toward the top of log.d.) Positional parameters look a bit odd but they are quite flexible. Andrei I don't think the positional parameters are odd (C# has them with another syntax and Tango uses the same syntax as C#), I think they're very usable. Although the syntax you've chosen looks a little odd to me. Wouldn't it make more sense to have the $ in front of the number instead of behind, since the % is in front. I used the Posix positional format syntax, see http://pubs.opengroup.org/onlinepubs/009695399/functions/printf.html. Andrei Ok, I see. -- /Jacob Carlborg
Re: Implementing std.log
On 2011-05-09 19:45, Andrei Alexandrescu wrote: On 5/9/11 12:24 PM, Jacob Carlborg wrote: I assume the initLogging function needs the application command line because you want to have the name of the application? If that's the case, I don't like that is handled via the application command line. I have a function that gets the path of the current process: http://dsource.org/projects/tango/attachment/ticket/1536/process.d This links to a file attached to a Tango ticket but don't worry, I've written the whole file myself and I intend to change the license to the Boost License. It's written with D1 (obviously) but it's very easy to port to D2, the only dependencies are C functions. Feel free to port it to D2 and use it in you're logging library if you want to. Thanks! In fact, the command-line arguments are needed for considerably more than just the application name. std.log is configurable via application's command line (e.g. --logtostderr --log_dir=/tmp/ etc., see documentation) and extracts these flags using getopt. The rest of the flags are left alone for the application to process. Yes, but since it requires you to pass an array I assumed the application name was the minimum requirement. In case the application would want to change a flag's name, e.g. use --log_to_stderr instead of --logtostderr, it's easy to define an API that does that; currently all parameter names are hoisted in private strings in std.log. Andrei -- /Jacob Carlborg
Re: Implementing std.log
On 09/05/2011 07:52, Andrei Alexandrescu wrote: On 5/7/11 3:43 PM, Jose Armando Garcia wrote: Hey folks, For the past couple of days I took the liberty of partially implementing a logging module for D. [snip] Nice work. Also thanks for contacting me offline. As I mentioned I'd already decided I'll take a stab at implementing a logging module inspired in design from glog. I was tired of the endless discussions on what a logging API should look like. This ironically is leading now to an embarrassment of riches - we now have two proposals on the table. I subjectively prefer mine for the simple reason that it includes exactly what I wanted from a logging subsystem with a light syntax. The documentation is at http://d-programming-language.org/phobos-prerelease/std_log.html. The source code is at http://erdani.com/log.d, with informative title only; to compile the code you'd need some changes in std.format as well (I extended the positional parameter syntax to allow ranges of positions). Todos include slight refactoring to avoid bloat in generated code per call, OS portability (I only tested on OSX), and getting threading right by fixing the shared-related compiler bugs. Looking forward to more discussion of the matter. Andrei The example looks great, exactly what I'd hope for in the way of a logging library. A few questions: * How do you disable logging at compile time? (and is it zero overhead?) * How do you define your own logging backend? * Is it possible to change logging options after initialisation? There are a couple of features missing, but you seem to have mentioned planning to implement these elsewhere in the thread. Once they're in place I'd be happy to vote yes to its inclusion into phobos. -- Robert http://octarineparrot.com/
Re: Implementing std.log
On 09/05/2011 19:18, Andrej Mitrovic wrote:
I'm not a big fan of that format syntax. It's just hard to look at and
figure out what went wrong when things do go wrong.
I'd prefer something like:
https://gist.github.com/963027
But someone would have to implement it, of course. I have my own
little echo() function which takes simple {} positional syntax, it
doesn't have any other formatting options though. But I only ever need
formatting for simple cases like the above. It seems some people need
a whole word processor macro language embedded in a format string..
That is A LOT nicer to look at. Please can we have this? :
--
Robert
http://octarineparrot.com/
Re: Implementing std.log
On 09/05/2011 19:30, Robert Clipsham wrote: The example looks great, exactly what I'd hope for in the way of a logging library. A few questions: * How do you disable logging at compile time? (and is it zero overhead?) * How do you define your own logging backend? * Is it possible to change logging options after initialisation? There are a couple of features missing, but you seem to have mentioned planning to implement these elsewhere in the thread. Once they're in place I'd be happy to vote yes to its inclusion into phobos. One more question: * Can you log using formatting, or do you have to wrap the call in a call to format()? eg. log.info(My %s string %s, format, 3); -- Robert http://octarineparrot.com/
Re: Implementing std.log
On 2011-05-09 20:03, Sean Kelly wrote: On May 9, 2011, at 10:45 AM, Andrei Alexandrescu wrote: On 5/9/11 12:24 PM, Jacob Carlborg wrote: I assume the initLogging function needs the application command line because you want to have the name of the application? If that's the case, I don't like that is handled via the application command line. I have a function that gets the path of the current process: http://dsource.org/projects/tango/attachment/ticket/1536/process.d This links to a file attached to a Tango ticket but don't worry, I've written the whole file myself and I intend to change the license to the Boost License. It's written with D1 (obviously) but it's very easy to port to D2, the only dependencies are C functions. Feel free to port it to D2 and use it in you're logging library if you want to. Thanks! In fact, the command-line arguments are needed for considerably more than just the application name. std.log is configurable via application's command line (e.g. --logtostderr --log_dir=/tmp/ etc., see documentation) and extracts these flags using getopt. The rest of the flags are left alone for the application to process. For what it's worth, the command-line args are available via core.runtime as well. Ah, you're right. -- /Jacob Carlborg
Re: Implementing std.log
On 2011-05-09 19:58, Andrei Alexandrescu wrote:
On 5/9/11 12:24 PM, Jacob Carlborg wrote:
I
have a function that gets the path of the current process:
http://dsource.org/projects/tango/attachment/ticket/1536/process.d
This links to a file attached to a Tango ticket but don't worry, I've
written the whole file myself and I intend to change the license to the
Boost License. It's written with D1 (obviously) but it's very easy to
port to D2, the only dependencies are C functions.
Feel free to port it to D2 and use it in you're logging library if you
want to.
I looked over the code, it's quite nice and clean. If you or anyone else
would want to create a github pull request, the function would be a
valuable addition to std.process. Thanks Jacob for offering the code to
Phobos!
What I think should change:
* Provide two overloads, one that attempts to reuse a buffer and one
that always returns a new string:
char[] getProcessPath(char[] buf) { ... }
static if (is(typeof(getProcessPath(null)) == char[]))
{
string getProcessPath() { return assumeUnique(getProcessPath(null)); }
}
Not sure I understand this.
* hoist version(OS) up to the function definition, e.g.
version (darwin) char[] getProcessPath (char[] buf) { ... }
version (freebsd) char[] getProcessPath (char[] buf) { ... }
...
Unsupported OSs will simply not define the function (as opposed to
asserting at run time).
Ok.
* Use size_t instead of uint throughout for 64-bit compatibility.
I'm just following the function signature, it's declared to take an
uint, like it or not:
http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man3/dyld.3.html
* Don't do this:
if (size buf.length)
buf ~= new char[size - buf.length];
because it allocates memory twice. Instead:
if (size buf.length)
buf.length = size;
which will allocate memory once or not at all.
* Check for all system calls for errors.
Again, thanks Jacob for your contribution!
Andrei
Ok, I'll see if I can find the time to create a pull request out of
this. In what module should I put the function?
--
/Jacob Carlborg
Re: Implementing std.log
On 2011-05-09 20:34, Robert Clipsham wrote:
On 09/05/2011 19:18, Andrej Mitrovic wrote:
I'm not a big fan of that format syntax. It's just hard to look at and
figure out what went wrong when things do go wrong.
I'd prefer something like:
https://gist.github.com/963027
But someone would have to implement it, of course. I have my own
little echo() function which takes simple {} positional syntax, it
doesn't have any other formatting options though. But I only ever need
formatting for simple cases like the above. It seems some people need
a whole word processor macro language embedded in a format string..
That is A LOT nicer to look at. Please can we have this? :
I like it as well, the same syntax used by C# and Tango.
--
/Jacob Carlborg
Re: Implementing std.log
On 5/9/11 1:30 PM, Robert Clipsham wrote: On 09/05/2011 07:52, Andrei Alexandrescu wrote: On 5/7/11 3:43 PM, Jose Armando Garcia wrote: Hey folks, For the past couple of days I took the liberty of partially implementing a logging module for D. [snip] Nice work. Also thanks for contacting me offline. As I mentioned I'd already decided I'll take a stab at implementing a logging module inspired in design from glog. I was tired of the endless discussions on what a logging API should look like. This ironically is leading now to an embarrassment of riches - we now have two proposals on the table. I subjectively prefer mine for the simple reason that it includes exactly what I wanted from a logging subsystem with a light syntax. The documentation is at http://d-programming-language.org/phobos-prerelease/std_log.html. The source code is at http://erdani.com/log.d, with informative title only; to compile the code you'd need some changes in std.format as well (I extended the positional parameter syntax to allow ranges of positions). Todos include slight refactoring to avoid bloat in generated code per call, OS portability (I only tested on OSX), and getting threading right by fixing the shared-related compiler bugs. Looking forward to more discussion of the matter. Andrei The example looks great, exactly what I'd hope for in the way of a logging library. A few questions: * How do you disable logging at compile time? (and is it zero overhead?) Passing -version=strip_log_error during compilation strips all error, warning, and info messages; passing -version=strip_log_warning strips all warning and info messages; and passing -version=strip_log_info strips all info messages. The critical and fatal logs cannot be stripped out. With -O -release -inline, log.info(arguments) leaves no trace in the generated object file. However, log.info.when(condition)(arguments) does leave a few calls behind. This is an issue in the inliner that will need to be fixed before releasing std.log. * How do you define your own logging backend? Currently the backend API is not exposed. The focus now is to get good client-level functionality in shape, after which we can work on a more refined API. We'll also need to tune the code to not generate undue bloating. * Is it possible to change logging options after initialisation? Not for the time being. I agree that that's a good thing to have. Everything is ready for exposure, so writing a couple of extra APIs should be easy. There are a couple of features missing, but you seem to have mentioned planning to implement these elsewhere in the thread. Once they're in place I'd be happy to vote yes to its inclusion into phobos. Sounds great! Andrei
Re: Implementing std.log
On 5/9/11 1:33 PM, Robert Clipsham wrote: On 09/05/2011 19:30, Robert Clipsham wrote: The example looks great, exactly what I'd hope for in the way of a logging library. A few questions: * How do you disable logging at compile time? (and is it zero overhead?) * How do you define your own logging backend? * Is it possible to change logging options after initialisation? There are a couple of features missing, but you seem to have mentioned planning to implement these elsewhere in the thread. Once they're in place I'd be happy to vote yes to its inclusion into phobos. One more question: * Can you log using formatting, or do you have to wrap the call in a call to format()? eg. log.info(My %s string %s, format, 3); I thought I'd leave that to format(), but probably it's worth just putting the feature in std.log too. One nice thing about supporting format directly is that often there's no need to allocate memory while logging. Andrei
Re: Implementing std.log
On 5/7/2011 1:43 PM, Jose Armando Garcia wrote: My intent, and hopefully we will get there with your help, is to include this in Phobos for D2. Thanks for doing the hard work of designing and laying out an implementation. Please, though, posting the source code as a news group message is not very efficient. Much better is to create an account with github, and post it there. Github has a lot of nice features that make collaboration, commenting, and incorporation easy, while a n.g. posting has none of that.
Re: Implementing std.log
On 5/9/11 1:48 PM, Jacob Carlborg wrote:
On 2011-05-09 19:58, Andrei Alexandrescu wrote:
On 5/9/11 12:24 PM, Jacob Carlborg wrote:
I
have a function that gets the path of the current process:
http://dsource.org/projects/tango/attachment/ticket/1536/process.d
This links to a file attached to a Tango ticket but don't worry, I've
written the whole file myself and I intend to change the license to the
Boost License. It's written with D1 (obviously) but it's very easy to
port to D2, the only dependencies are C functions.
Feel free to port it to D2 and use it in you're logging library if you
want to.
I looked over the code, it's quite nice and clean. If you or anyone else
would want to create a github pull request, the function would be a
valuable addition to std.process. Thanks Jacob for offering the code to
Phobos!
What I think should change:
* Provide two overloads, one that attempts to reuse a buffer and one
that always returns a new string:
char[] getProcessPath(char[] buf) { ... }
static if (is(typeof(getProcessPath(null)) == char[]))
{
string getProcessPath() { return assumeUnique(getProcessPath(null)); }
}
Not sure I understand this.
The static if actually refers to the next point, which implies that
getProcessPath(char[]) may not exist on some systems.
The idea behind having two functions is to give a string to people who
just want a string without messing with buffers and all. The string has
the nice property that nobody can trample on it later.
Ok, I'll see if I can find the time to create a pull request out of
this. In what module should I put the function?
It looks like core.runtime already allows fetching the process path
name, so you need to figure out what the added value of your function is.
Andrei
Re: Implementing std.log
On 5/9/11 1:34 PM, Robert Clipsham wrote:
On 09/05/2011 19:18, Andrej Mitrovic wrote:
I'm not a big fan of that format syntax. It's just hard to look at and
figure out what went wrong when things do go wrong.
I'd prefer something like:
https://gist.github.com/963027
But someone would have to implement it, of course. I have my own
little echo() function which takes simple {} positional syntax, it
doesn't have any other formatting options though. But I only ever need
formatting for simple cases like the above. It seems some people need
a whole word processor macro language embedded in a format string..
That is A LOT nicer to look at. Please can we have this? :
I, too, prefer it a great deal to the prinf/scanf format, and it
wouldn't be difficult to implement, but at this point we need to worry
about compatibility issues and such (TDPL already uses the % syntax as
does existing D code). I don't see a simple migration path.
One possibility is to provide a nice2ugly template that transforms C#
format strings into % format strings, e.g. nice2ugly!{1}:{2} yields
the string constant %1$s:%2$s.
By and large this doesn't seem to be a very worrisome issue as
formatting strings are a minority in most applications and are seldom
maintained. You pay attention, get them right, and then just use them.
Andrei
Re: Implementing std.log
Andrei Alexandrescu wrote: On 5/7/11 3:43 PM, Jose Armando Garcia wrote: Hey folks, For the past couple of days I took the liberty of partially implementing a logging module for D. [snip] Nice work. Also thanks for contacting me offline. As I mentioned I'd already decided I'll take a stab at implementing a logging module inspired in design from glog. I was tired of the endless discussions on what a logging API should look like. This ironically is leading now to an embarrassment of riches - we now have two proposals on the table. I subjectively prefer mine for the simple reason that it includes exactly what I wanted from a logging subsystem with a light syntax. The documentation is at http://d-programming-language.org/phobos-prerelease/std_log.html. The source code is at http://erdani.com/log.d, with informative title only; to compile the code you'd need some changes in std.format as well (I extended the positional parameter syntax to allow ranges of positions). Todos include slight refactoring to avoid bloat in generated code per call, OS portability (I only tested on OSX), and getting threading right by fixing the shared-related compiler bugs. Looking forward to more discussion of the matter. I think every() behaves strangely. Because the counter is per function. But it should be per logging statement. std.log differs here from glog and this seems incorrect to me. everyMs() has a similar problem. Even though I believe log.vlog is necessary I often find it difficult to come up with an appropriate level for a given statement. With normal logging it is often obvious whether info, warning, error, critical, or fatal is appropriate. But this is a minor problem and more a matter of coding style. I'd like to try out the code. I'd be very pleased if you can make a compiling version available on github. I'd like to test whether there is no code generated if -version=strip_log_error etc. is specified. I.e. whether the call to an empty function will always be removed. Jens
Re: Implementing std.log
On 09/05/2011 20:47, Andrei Alexandrescu wrote:
On 5/9/11 1:34 PM, Robert Clipsham wrote:
That is A LOT nicer to look at. Please can we have this? :
I, too, prefer it a great deal to the prinf/scanf format, and it
wouldn't be difficult to implement, but at this point we need to worry
about compatibility issues and such (TDPL already uses the % syntax as
does existing D code). I don't see a simple migration path.
One possibility is to provide a nice2ugly template that transforms C#
format strings into % format strings, e.g. nice2ugly!{1}:{2} yields
the string constant %1$s:%2$s.
By and large this doesn't seem to be a very worrisome issue as
formatting strings are a minority in most applications and are seldom
maintained. You pay attention, get them right, and then just use them.
Andrei
Compilation switch maybe? Support both by default, compile with
-version=DisableUglyFormatStrings to remove the overhead. Document it as
deprecated, tweak the first revision of TDPL to use the new format
strings, give it X amount of time, make it deprecated etc. Having
nice2ugly! all over my codebase is not a solution.
--
Robert
http://octarineparrot.com/
Re: Implementing std.log
On 5/9/11, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote:
One possibility is to provide a nice2ugly template that transforms C#
format strings into % format strings, e.g. nice2ugly!{1}:{2} yields
the string constant %1$s:%2$s.
That is exactly what my echo function does. It replaces {} syntax with
%$s syntax behind the scenes, and then forwards that to writef.
Re: Implementing std.log
On Sat, May 7, 2011 at 11:03 PM, dsimcha dsim...@yahoo.com wrote: Ok, there's clearly been some misunderstanding here. My real point was, why do you need this threading at all? It's definitely overkill for a single threaded application, but for things like the application I'm working on, which is multithreaded and already uses message passing between threads, I think it would fit in quite nicely. My question would be whether it's pluggable enough to allow the simple case of a single threaded program that doesn't need message passing to be implemented.
