Yes, you can filter at collecting time or appending time. Up to you.
Regards JB On 24/06/2019 04:04, Noobtube Account wrote: > Looks like what I’m looking for can be done using collector not appender. > > org.apache.karaf.decanter.collector.log.cfg: > fields.remove.renderedMessage=true > > Cheers! > > On Mon, 24 Jun 2019 at 11:54 am, Noobtube Account <spidyt...@gmail.com > <mailto:spidyt...@gmail.com>> wrote: > > Thanks JB - I think the EventFilter and the properties along with it > are used to include/exclude entire events, correct? > > In that case, I’m not able to use it to exclude fields of the > resulting json log but instead exclude entire logs based on the > content of the log message. > > I guess I’ll need to put this functionality in a custom marshaller? > > > Thanks > > On Fri, 21 Jun 2019 at 5:49 pm, Jean-Baptiste Onofré > <j...@nanthrax.net <mailto:j...@nanthrax.net>> wrote: > > Hi, > > I guess you are talking about filtering of the events to send to the > appender right ? > > I will update the Decanter documentation with some example and > how it works. > > In the mean time, you can take a look on the test illustrating the > events filtering: > > > https://github.com/apache/karaf-decanter/blob/master/appender/utils/src/test/java/org/apache/karaf/decanter/appender/utils/EventFilterTest.java > > Regards > JB > > On 21/06/2019 09:41, Noobtube Account wrote: > > Hi JB, > > > > Could you show me where I could add the filter properties as you > > mentioned or if it’s not implemented yet, some guidance on > accessing the > > appender or Marshaller properties file from inside the default > decanter > > json appender? > > > > > > Thanks > > > > > > > > On Wed, 12 Jun 2019 at 11:53 am, Noobtube Account > <spidyt...@gmail.com <mailto:spidyt...@gmail.com> > > <mailto:spidyt...@gmail.com <mailto:spidyt...@gmail.com>>> wrote: > > > > Thanks JB! I've got the following config for appender to > try out the > > field exclude: > > > > # File > > filename=${karaf.data}/log/karaf-json.log > > > > # Marshaller > > marshaller.target=(dataFormat=json) > > > > # Append file > > append=true > > > > # Fields Excluded by Name > > event.property.name.exclude=MDC > > > > > > This seems to stop the appender from producing any json > log. Removing the event.property.name.exclude=MDC > > > > Also I couldn't find these properties documented anywhere > nor could I find them in karaf decanter source code. Is the > format I used correct? > > > > > > > > On Sun, 9 Jun 2019 at 02:58, Jean-Baptiste Onofré > <j...@nanthrax.net <mailto:j...@nanthrax.net> > > <mailto:j...@nanthrax.net <mailto:j...@nanthrax.net>>> wrote: > > > > Yes, you can create your own marshaller is you want to ;) > > > > By default, Decanter collectors send a Map to the Decanter > > appenders. > > Then the appenders use eventually marshaller to > convert the Map to > > something else. You can control the keys or values in > this map > > that you > > want to include/exclude by directly configuring in the > appender > > configuration (with > > > > event.property.name.exclude/event.property.name.include/event.property.value.exclude/event.property.value.include > > properties). > > By default, the JSON Marshaller convert the map as a > json using > > jackson. > > If you want a full control, you can also create your > own Marshaller > > service. But I think that just the filtering on the > appender > > should be > > OK for you. > > Let me know if you need help about that. > > > > Regards > > JB > > > > On 08/06/2019 18:52, Noobtube Account wrote: > > > Great! Are we also able to control the format or > turn fields off? > > > > > > > > > On Sun, 9 Jun 2019 at 2:51 am, Jean-Baptiste Onofré > > <j...@nanthrax.net <mailto:j...@nanthrax.net> > <mailto:j...@nanthrax.net <mailto:j...@nanthrax.net>> > > > <mailto:j...@nanthrax.net <mailto:j...@nanthrax.net> > <mailto:j...@nanthrax.net <mailto:j...@nanthrax.net>>>> wrote: > > > > > > Yes, Decanter Log Collector is a global Pax Logging > > "sniffer". So it > > > takes messages from any bundle by default. > > > > > > However, you can configure this directly in > > > etc/org.ops4j.pax.logging.cfg. > > > > > > Regards > > > JB > > > > > > On 08/06/2019 18:07, Noobtube Account wrote: > > > > Oh, scratch that. Uncommenting the filename in > > > > > etc/org.apache.karaf.decanter.appender.file.cfg seemed > > to have made it > > > > work. It probably wasn't due to that though > and likely > > something > > > else I > > > > did. Anyway, thanks for pointing out karaf > decanter. > > > > > > > > Will the log collector collect log executions made > > through any > > > bundle? > > > > > > > > > > > > On Sun, 9 Jun 2019 at 01:55, Noobtube Account > > <spidyt...@gmail.com <mailto:spidyt...@gmail.com> > <mailto:spidyt...@gmail.com <mailto:spidyt...@gmail.com>> > > > <mailto:spidyt...@gmail.com > <mailto:spidyt...@gmail.com> <mailto:spidyt...@gmail.com > <mailto:spidyt...@gmail.com>>> > > > > <mailto:spidyt...@gmail.com > <mailto:spidyt...@gmail.com> <mailto:spidyt...@gmail.com > <mailto:spidyt...@gmail.com>> > > <mailto:spidyt...@gmail.com > <mailto:spidyt...@gmail.com> <mailto:spidyt...@gmail.com > <mailto:spidyt...@gmail.com>>>>> wrote: > > > > > > > > Thanks JB. I'm trying the decanter as you > mentioned. > > > > > > > > I've done: > > > > feature:repo-add decanter > > > > feature:install decanter-collector-log > > > > feature:install decanter-appender-file > > > > > > > > Then I edited > > etc/org.apache.karaf.decanter.appender.file.cfg and > > > > added: > > > > # Marshaller > > > > marshaller.target=(dataFormat=json) > > > > > > > > But I don't see any logs created. Do we > need to > > install the > > > marshaller? > > > > > > > > On Sun, 9 Jun 2019 at 00:33, Jean-Baptiste > Onofré > > > <j...@nanthrax.net <mailto:j...@nanthrax.net> > <mailto:j...@nanthrax.net <mailto:j...@nanthrax.net>> > > <mailto:j...@nanthrax.net <mailto:j...@nanthrax.net> > <mailto:j...@nanthrax.net <mailto:j...@nanthrax.net>>> > > > > <mailto:j...@nanthrax.net > <mailto:j...@nanthrax.net> <mailto:j...@nanthrax.net > <mailto:j...@nanthrax.net>> > > <mailto:j...@nanthrax.net <mailto:j...@nanthrax.net> > <mailto:j...@nanthrax.net <mailto:j...@nanthrax.net>>>>> wrote: > > > > > > > > By the way, instead of doing that, you > can use > > regular logging > > > > and use > > > > Decanter with the log collector and > the json > > marshaller to > > > > create json > > > > output (to log, file, elasticsearch or > whatever). > > > > > > > > Regards > > > > JB > > > > > > > > On 08/06/2019 16:30, Jean-Baptiste > Onofré wrote: > > > > > Hi > > > > > > > > > > You don't have to add logback-core as > > pax-logging-logback > > > > bundle already > > > > > embed it. > > > > > > > > > > Secondly it seems jackson is not > imported > > correctly. The > > > root > > > > cause is: > > > > > > > > > > Caused by: > java.lang.ClassNotFoundException: > > > > >> > com.fasterxml.jackson.databind.ObjectMapper > > not found by > > > > >> > org.ops4j.pax.logging.pax-logging-logback > > > > > > > > > > You can try a dynamic import on > > pax-logging-logback. > > > > > > > > > > Else a simple workaround is to create a > > fragment with > > > > > pax-logging-logback as host and > embedding > > (private-package) > > > > the classes > > > > > you need. > > > > > > > > > > I did that for json layout with > > pax-logging-log4j2. > > > > > > > > > > Regards > > > > > JB > > > > > > > > > > On 08/06/2019 16:17, Noobtube > Account wrote: > > > > >> I'm trying to get JSON format > logging enabled > > in Karaf. > > > > Trying to do > > > > >> something similar to what's described > > > > >> > in https://ops4j1.jira.com/browse/PAXLOGGING-168 > > > > >> > > > > >> On a fresh copy of Karaf 4.2.5 I've > added to > > > > /etc/startup.properties: > > > > >> > > > > >> > > mvn\:com.fasterxml.jackson.core/jackson-core/2.9.5 = 8 > > > > >> > > > > mvn\:com.fasterxml.jackson.core/jackson-annotations/2.9.5 = 8 > > > > >> > > mvn\:com.fasterxml.jackson.core/jackson-databind/2.9.5 = 8 > > > > >> > mvn\:ch.qos.logback/logback-classic/1.2.2 = 8 > > > > >> > mvn\:ch.qos.logback/logback-core/1.2.2 = 8 > > > > >> > > mvn\:ch.qos.logback.contrib/logback-json-core/0.1.5 = 8 > > > > >> > > mvn\:ch.qos.logback.contrib/logback-json-classic/0.1.5 = 8 > > > > >> > > mvn\:org.ops4j.pax.logging/pax-logging-logback/1.10.1 = 8 > > > > >> > > > > >> The above jars have been added to > the system > > folder. They > > > > look like: > > > > >> > logback: https://pastebin.com/raw/60Pnu8f1 > > > > >> > jackson: https://pastebin.com/raw/NkUqZrGS > > > > >> > > > > >> My etc/org.ops4j.pax.logging.cfg > contains the > > line: > > > > >> > > > > > > > > > > > org.ops4j.pax.logging.logback.config.file=${karaf.base}/etc/logback.xml > > > > >> > > > > >> The log back is the same one > > > > >> > > from https://ops4j1.jira.com/browse/PAXLOGGING-168 > > > > >> > > > > >> When I startup Karaf I get the > following CNF > > error > > > related to > > > > Jackson. > > > > >> Shouldn't the startup.properties > file make > > those classes > > > > available? > > > > >> > > > > >> WARN in > > > > > > > ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5c3aaee6 - > > > > >> SizeAndTimeBasedFNATP is > deprecated. Use > > > > SizeAndTimeBasedRollingPolicy > > > > >> instead > > > > >> WARN in > > > > > > > ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@5c3aaee6 > > > - For > > > > >> more information see > > > > >> > > > > > > > > > > > http://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy > > > > >> 00:14:49.839 [CM Configuration Updater > > (ManagedService > > > Update: > > > > >> pid=[org.ops4j.pax.logging])] ERROR > > > > org.apache.felix.configadmin - > > > > >> [org.osgi.service.log.LogService, > > > > >> > org.knopflerfish.service.log.LogService, > > > > >> > org.ops4j.pax.logging.PaxLoggingService, > > > > >> org.osgi.service.cm > <http://org.osgi.service.cm> > > <http://org.osgi.service.cm> <http://org.osgi.service.cm> > > > > > <http://org.osgi.service.cm>.ManagedService, id=15, > > > > >> > > > > > > bundle=13/mvn:org.ops4j.pax.logging/pax-logging-logback/1.10.1]: > > > > >> Unexpected problem updating > configuration > > > org.ops4j.pax.logging > > > > >> java.lang.NoClassDefFoundError: > > > > > com/fasterxml/jackson/databind/ObjectMapper > > > > >> at > > > > >> > > > > > > > > > > > ch.qos.logback.contrib.jackson.JacksonJsonFormatter.<init>(JacksonJsonFormatter.java:41) > > > > >> ~[?:?] > > > > >> at > > > > > > > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > > > > Method) > > > > >> ~[?:?] > > > > >> at > > > > >> > > > > > > > > > > > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > > > > >> ~[?:?] > > > > >> at > > > > >> > > > > > > > > > > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > > > > >> ~[?:?] > > > > >> at > > > > > > > > > > java.lang.reflect.Constructor.newInstance(Constructor.java:423) > > > > ~[?:?] > > > > >> at > > java.lang.Class.newInstance(Class.java:442) ~[?:?] > > > > >> at > > > > >> > > > > > > > > > > > ch.qos.logback.core.joran.action.NestedComplexPropertyIA.begin(NestedComplexPropertyIA.java:121) > > > > >> ~[?:?] > > > > >> at > > > > >> > > > > > > > > > > > ch.qos.logback.core.joran.spi.Interpreter.callBeginAction(Interpreter.java:269) > > > > >> ~[?:?] > > > > >> at > > > > >> > > > > > > > > > > > ch.qos.logback.core.joran.spi.Interpreter.startElement(Interpreter.java:145) > > > > >> ~[?:?] > > > > >> at > > > > >> > > > > > > > > > > > ch.qos.logback.core.joran.spi.Interpreter.startElement(Interpreter.java:128) > > > > >> ~[?:?] > > > > >> at > > > > > > > > > > ch.qos.logback.core.joran.spi.EventPlayer.play(EventPlayer.java:50) > > > > >> ~[?:?] > > > > >> at > > > > >> > > > > > > > > > > > ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:165) > > > > >> ~[?:?] > > > > >> at > > > > >> > > > > > > > > > > > ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:152) > > > > >> ~[?:?] > > > > >> at > > > > >> > > > > > > > > > > > ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:110) > > > > >> ~[?:?] > > > > >> at > > > > >> > > > > > > > > > > > ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:81) > > > > >> ~[?:?] > > > > >> at > > > > >> > > > > > > > > > > > org.ops4j.pax.logging.logback.internal.PaxLoggingServiceImpl.configureLogback(PaxLoggingServiceImpl.java:325) > > > > >> ~[?:?] > > > > >> at > > > > >> > > > > > > > > > > > org.ops4j.pax.logging.logback.internal.PaxLoggingServiceImpl.updated(PaxLoggingServiceImpl.java:258) > > > > >> ~[?:?] > > > > >> at > > > > >> > > > > > > > > > > > org.ops4j.pax.logging.logback.internal.PaxLoggingServiceImpl$1ManagedPaxLoggingService.updated(PaxLoggingServiceImpl.java:471) > > > > >> ~[?:?] > > > > >> at > > > > >> > > > > > > > > > > > org.apache.felix.cm.impl.helper.ManagedServiceTracker.updated(ManagedServiceTracker.java:189) > > > > >> > ~[9:org.apache.felix.configadmin:1.9.14] > > > > >> at > > > > >> > > > > > > > > > > > org.apache.felix.cm.impl.helper.ManagedServiceTracker.updateService(ManagedServiceTracker.java:152) > > > > >> [9:org.apache.felix.configadmin:1.9.14] > > > > >> at > > > > >> > > > > > > > > > > > org.apache.felix.cm.impl.helper.ManagedServiceTracker.provideConfiguration(ManagedServiceTracker.java:85) > > > > >> [9:org.apache.felix.configadmin:1.9.14] > > > > >> at > > > > >> > > > > > > > > > > > org.apache.felix.cm.impl.ConfigurationManager$ManagedServiceUpdate.provide(ConfigurationManager.java:1113) > > > > >> [9:org.apache.felix.configadmin:1.9.14] > > > > >> at > > > > >> > > > > > > > > > > > org.apache.felix.cm.impl.ConfigurationManager$ManagedServiceUpdate.run(ConfigurationManager.java:1069) > > > > >> [9:org.apache.felix.configadmin:1.9.14] > > > > >> at > > > > > > > > > > org.apache.felix.cm.impl.UpdateThread.run0(UpdateThread.java:138) > > > > >> [9:org.apache.felix.configadmin:1.9.14] > > > > >> at > > > > > > > > > > org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:105) > > > > >> [9:org.apache.felix.configadmin:1.9.14] > > > > >> at > java.lang.Thread.run(Thread.java:748) [?:?] > > > > >> Caused by: > java.lang.ClassNotFoundException: > > > > >> > com.fasterxml.jackson.databind.ObjectMapper > > not found by > > > > >> > org.ops4j.pax.logging.pax-logging-logback [13] > > > > >> at > > > > >> > > > > > > > > > > > org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1639) > > > > >> ~[?:?] > > > > >> at > > > > >> > > > > > > > > > > > org.apache.felix.framework.BundleWiringImpl.access$200(BundleWiringImpl.java:80) > > > > >> ~[?:?] > > > > >> at > > > > >> > > > > > > > > > > > org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:2053) > > > > >> ~[?:?] > > > > >> at > > > > java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:?] > > > > >> ... 26 more > > > > >> > > > > >> > > > > > > > > > > > > > -- > > > > Jean-Baptiste Onofré > > > > jbono...@apache.org > <mailto:jbono...@apache.org> <mailto:jbono...@apache.org > <mailto:jbono...@apache.org>> > > <mailto:jbono...@apache.org > <mailto:jbono...@apache.org> <mailto:jbono...@apache.org > <mailto:jbono...@apache.org>>> > > > <mailto:jbono...@apache.org > <mailto:jbono...@apache.org> <mailto:jbono...@apache.org > <mailto:jbono...@apache.org>> > > <mailto:jbono...@apache.org > <mailto:jbono...@apache.org> <mailto:jbono...@apache.org > <mailto:jbono...@apache.org>>>> > > > > http://blog.nanthrax.net > > > > Talend - http://www.talend.com > > > > > > > > > > -- > > > Jean-Baptiste Onofré > > > jbono...@apache.org <mailto:jbono...@apache.org> > <mailto:jbono...@apache.org <mailto:jbono...@apache.org>> > > <mailto:jbono...@apache.org > <mailto:jbono...@apache.org> <mailto:jbono...@apache.org > <mailto:jbono...@apache.org>>> > > > http://blog.nanthrax.net > > > Talend - http://www.talend.com > > > > > > > -- > > Jean-Baptiste Onofré > > jbono...@apache.org <mailto:jbono...@apache.org> > <mailto:jbono...@apache.org <mailto:jbono...@apache.org>> > > http://blog.nanthrax.net > > Talend - http://www.talend.com > > > > -- > Jean-Baptiste Onofré > jbono...@apache.org <mailto:jbono...@apache.org> > http://blog.nanthrax.net > Talend - http://www.talend.com > -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com
