JsonTemplateLayout for Google Cloud Logging
I was thinking about contributing a JsonTemplateLayout for the structured logging format expected by Google Cloud Logging [1]. This is what I've come up with so far: any comments are welcome: https://gist.github.com/rocketraman/824ca9527bb8c7226d40db2b4d78efa7. [1] https://cloud.google.com/logging/docs/structured-logging Regards, Raman
Re: Log4Go
Well, I am on board with part of what you want. When I looked at various Go logging libraries most of them defined where the logs ere going and how they are formatted at build time. 90% of what Log4j does is figure out all of that at run time. Go is not nearly as dynamic as Java because the application is constructed completely at build time. You can’t just add libraries and have the behavior change without rebuilding. However, it is still possible to include all the things you want your application to be able to choose from and then bind to a specific component via configuration. So it should still be possible to implement much of the flexibility that Log4j provides. One problem I did notice though is that there are several popular logging libraries. Due to the nature of Go it isn’t really possible to replace them with a bridge to the logging framework you really want to use. To be honest, I have no idea how Go applications deal with having dependencies that all use different logging libraries. Ralph > On Jun 30, 2021, at 9:12 AM, Christofer Dutz wrote: > > Hi all, > > and sorry for being late to the party ;-) > > I am currently working hard on PLC4X' Go support and am also using what I > create in the Open-Source project in some larger corporate applications. > > One thing that has always bugged me with go, was the inavailability of > loggers that allow me to set different log levels for different parts of the > application. In go with every half-efficient logging framework, it's an all > or nothing thing. So if I want to track down a problem in my driver for > protocol X and I switch logging to TRACE it's like trying to drink out of an > open fire-hose. > > What I would love to do as a first step, and I don't think it should be too > complicated, would be to create a Go API that allows us to define hierarchies > of log levels, just like we know them in the Java world. This API would be > used in the application to log, but it wouldn't actually do any logging but > internally sort of use an underlying framework (possibly auto configured to > TRACE or the most talkative log level) and forward log requests to that if it > passes the filter criteria. > > So in PLC4Go for example we could use this Go Logging API. If my company now > uses logrus or zerolog, then all we have to do in that application is > initialize the log4go system (I know there's a project using that name > pattern ... I'm referring to something we built) with the corresponding > adapter. > > What do you think? I'm not one of these "I whish someone would build X for > me" folks ... I am willing to put quite some effort into something like this. > But I think it should be in a project like Apache Logging and not as a side > project of PLC4X. > > Chris > > > On 2020/12/11 12:20:18 Volkan Yazıcı wrote: >> I support the initiative. At bol.com, we also needed to implement our own >> Go logging layouts (JSON) and appenders (Redis). That said, I don't know Go >> and I don't think I will be able to spare time to both learn a new language >> (even though I am really into learning Go) and maintain such a project. I >> mean, not that you need my help, but just wanted to share my availability. >> If I would have time, I would rather clean up Log4j bugs piled up in JIRA. >> >> I also agree with Matt that this would pave the road to standardizing the >> logging configuration file formats across multiple languages. >> >> What I witness most for code — in particular libraries, APIs, etc. — >> written by programmers whose expertise is actually in another language, >> that they mostly don't get the language conventions right. For instance, I >> was horrified many times in the past to read/use Java code written by >> JavaScript (front-end) developers. These two languages have totally >> different approaches and (community embraced) conventions that one cannot >> plug-n-play the mindset of one to another. In conclusion, as far as I know, >> none of us is programming in Go on a daily basis. Hence, I strongly >> recommend consulting to experts in this domain before publishing something >> to the outside world. For one, I am pretty sure there should be Go experts >> within the Apache community, hence having expert reviews should be >> relatively easy. Second, Apache has such a good track record in delivering >> high quality software, even an inferior project might get quite some >> attraction and we will be bound to maintain it for years. These are my >> concerns in general. That said, I would be more than happy to ditch off our >> custom Go loggers with an Apache-approved alternative. >> >> On Wed, Dec 9, 2020 at 10:29 PM Ralph Goers >> wrote: >> >>> The company I work for has started using Go for some of the middleware >>> components we are developing. I have looked at several logging frameworks >>> for Go and have not been impressed by any of them. As such, I am >>> considering starting a project
Re: Re: Log4Go
I'm rather new to go, but looking for ways to improve by writing code alongside people who actually know what they're doing. If I can help, please ping me. -d On June 30, 2021 18:12:46 Christofer Dutz wrote: Hi all, and sorry for being late to the party ;-) I am currently working hard on PLC4X' Go support and am also using what I create in the Open-Source project in some larger corporate applications. One thing that has always bugged me with go, was the inavailability of loggers that allow me to set different log levels for different parts of the application. In go with every half-efficient logging framework, it's an all or nothing thing. So if I want to track down a problem in my driver for protocol X and I switch logging to TRACE it's like trying to drink out of an open fire-hose. What I would love to do as a first step, and I don't think it should be too complicated, would be to create a Go API that allows us to define hierarchies of log levels, just like we know them in the Java world. This API would be used in the application to log, but it wouldn't actually do any logging but internally sort of use an underlying framework (possibly auto configured to TRACE or the most talkative log level) and forward log requests to that if it passes the filter criteria. So in PLC4Go for example we could use this Go Logging API. If my company now uses logrus or zerolog, then all we have to do in that application is initialize the log4go system (I know there's a project using that name pattern ... I'm referring to something we built) with the corresponding adapter. What do you think? I'm not one of these "I whish someone would build X for me" folks ... I am willing to put quite some effort into something like this. But I think it should be in a project like Apache Logging and not as a side project of PLC4X. Chris On 2020/12/11 12:20:18 Volkan Yazıcı wrote: I support the initiative. At bol.com, we also needed to implement our own Go logging layouts (JSON) and appenders (Redis). That said, I don't know Go and I don't think I will be able to spare time to both learn a new language (even though I am really into learning Go) and maintain such a project. I mean, not that you need my help, but just wanted to share my availability. If I would have time, I would rather clean up Log4j bugs piled up in JIRA. I also agree with Matt that this would pave the road to standardizing the logging configuration file formats across multiple languages. What I witness most for code — in particular libraries, APIs, etc. — written by programmers whose expertise is actually in another language, that they mostly don't get the language conventions right. For instance, I was horrified many times in the past to read/use Java code written by JavaScript (front-end) developers. These two languages have totally different approaches and (community embraced) conventions that one cannot plug-n-play the mindset of one to another. In conclusion, as far as I know, none of us is programming in Go on a daily basis. Hence, I strongly recommend consulting to experts in this domain before publishing something to the outside world. For one, I am pretty sure there should be Go experts within the Apache community, hence having expert reviews should be relatively easy. Second, Apache has such a good track record in delivering high quality software, even an inferior project might get quite some attraction and we will be bound to maintain it for years. These are my concerns in general. That said, I would be more than happy to ditch off our custom Go loggers with an Apache-approved alternative. On Wed, Dec 9, 2020 at 10:29 PM Ralph Goers wrote: > The company I work for has started using Go for some of the middleware > components we are developing. I have looked at several logging frameworks > for Go and have not been impressed by any of them. As such, I am > considering starting a project here. The major goals of this would be: > > Use an external configuration (at least JSON and XML). > Allow the configuration to be accessed via HTTP(S) - Spring Cloud > Configuration. > Allow dynamic reconfiguration. > Allow plugins (probably as Go plugins?) > Support for Markers, context attributes, Layouts, Appenders. > > Anyone interested? > > Ralph > >
Re: Re: Log4Go
Hi all, and sorry for being late to the party ;-) I am currently working hard on PLC4X' Go support and am also using what I create in the Open-Source project in some larger corporate applications. One thing that has always bugged me with go, was the inavailability of loggers that allow me to set different log levels for different parts of the application. In go with every half-efficient logging framework, it's an all or nothing thing. So if I want to track down a problem in my driver for protocol X and I switch logging to TRACE it's like trying to drink out of an open fire-hose. What I would love to do as a first step, and I don't think it should be too complicated, would be to create a Go API that allows us to define hierarchies of log levels, just like we know them in the Java world. This API would be used in the application to log, but it wouldn't actually do any logging but internally sort of use an underlying framework (possibly auto configured to TRACE or the most talkative log level) and forward log requests to that if it passes the filter criteria. So in PLC4Go for example we could use this Go Logging API. If my company now uses logrus or zerolog, then all we have to do in that application is initialize the log4go system (I know there's a project using that name pattern ... I'm referring to something we built) with the corresponding adapter. What do you think? I'm not one of these "I whish someone would build X for me" folks ... I am willing to put quite some effort into something like this. But I think it should be in a project like Apache Logging and not as a side project of PLC4X. Chris On 2020/12/11 12:20:18 Volkan Yazıcı wrote: > I support the initiative. At bol.com, we also needed to implement our own > Go logging layouts (JSON) and appenders (Redis). That said, I don't know Go > and I don't think I will be able to spare time to both learn a new language > (even though I am really into learning Go) and maintain such a project. I > mean, not that you need my help, but just wanted to share my availability. > If I would have time, I would rather clean up Log4j bugs piled up in JIRA. > > I also agree with Matt that this would pave the road to standardizing the > logging configuration file formats across multiple languages. > > What I witness most for code — in particular libraries, APIs, etc. — > written by programmers whose expertise is actually in another language, > that they mostly don't get the language conventions right. For instance, I > was horrified many times in the past to read/use Java code written by > JavaScript (front-end) developers. These two languages have totally > different approaches and (community embraced) conventions that one cannot > plug-n-play the mindset of one to another. In conclusion, as far as I know, > none of us is programming in Go on a daily basis. Hence, I strongly > recommend consulting to experts in this domain before publishing something > to the outside world. For one, I am pretty sure there should be Go experts > within the Apache community, hence having expert reviews should be > relatively easy. Second, Apache has such a good track record in delivering > high quality software, even an inferior project might get quite some > attraction and we will be bound to maintain it for years. These are my > concerns in general. That said, I would be more than happy to ditch off our > custom Go loggers with an Apache-approved alternative. > > On Wed, Dec 9, 2020 at 10:29 PM Ralph Goers > wrote: > > > The company I work for has started using Go for some of the middleware > > components we are developing. I have looked at several logging frameworks > > for Go and have not been impressed by any of them. As such, I am > > considering starting a project here. The major goals of this would be: > > > > Use an external configuration (at least JSON and XML). > > Allow the configuration to be accessed via HTTP(S) - Spring Cloud > > Configuration. > > Allow dynamic reconfiguration. > > Allow plugins (probably as Go plugins?) > > Support for Markers, context attributes, Layouts, Appenders. > > > > Anyone interested? > > > > Ralph > > > > >
