Take a look at ReportingTask, this sounds like a good fit. Much higher level than explicitly putting log steps in the flow or reconfiguring the logging system to forward a category elsewhere.
HTH, Andrew [1] https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#reporting-tasks From: sudeep mishra <sudeepshekh...@gmail.com<mailto:sudeepshekh...@gmail.com>> Reply-To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Date: Wednesday, February 10, 2016 at 11:59 PM To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Subject: Re: NiFi logging Thanks for your feedback Andy. I am looking to log the general events taking place in data flow to track my data at a later stage. Say I have below flow: Flow 1 - User sending XML data over HTTP POST --> NiFi HTTPListener --> NiFi PutKafka processor Flow 2 - NiFi GetKafka processor --> NiFi UpdateAttribute processor <include some metaadta> --> NiFi RouteOnAttribute processor --> NiFi PutHDFS processor For each such XML posted by a user I want to log the time and some attributes for the XML received at each flow. Such as <Time> <NiFi HTTPListener> Received <X> XML at Listener <Time> < NiFi PutKafka processor> <X> XML moved to topic <KafkaTopic> <Time> < NiFi GettKafka processor> Received <X> XML moved from topic <KafkaTopic> ...... <Time> < NiFi PutHDFS processor> Moved <X> XML moved to HDFS On Thu, Feb 11, 2016 at 10:13 AM, Andy LoPresto <alopresto.apa...@gmail.com<mailto:alopresto.apa...@gmail.com>> wrote: Hi Sudeep. SLF4J Loggers are available throughout the code. Are you trying to log details from inside a processor? Within the framework, there are various implementations of `ComponentLog` [1], and `ProcessorLog` (just a marker interface), which is implemented by `SimpleProcessLogger` [2]. From within a processor which extends `AbstractProcessor` [3], you can call `getLogger()` to return an instance of `ProcessorLog` and then write to it as you would any SLF4J logger instance. ```java String key = "someKey"; String value = "someValue"; final ProcessorLog logger = getLogger(); logger.warn("This is a warning message"); logger.info<http://logger.info>("This is a message with some interpolation key={}, value={}", new String[]{key, value}); ``` This will result in the output: ``` [pool-2-thread-1] WARN org.apache.nifi.processors.standard.EncryptContent - EncryptContent[id=1c569c19-d37b-4d30-aec7-410b318c360d] This is a warning message [pool-2-thread-1] INFO org.apache.nifi.processors.standard.EncryptContent - EncryptContent[id=1c569c19-d37b-4d30-aec7-410b318c360d] This is a message with some interpolation key=someKey, value=someValue ``` We recommend using SLF4J, but if you have a requirement to use Log4j, you can use an adapter as described here [4]. [1] https://github.com/apache/nifi/blob/3a7ddc6a35f143c1d4a735226fe7467a83d04583/nifi-api/src/main/java/org/apache/nifi/logging/ComponentLog.java [2] https://github.com/apache/nifi/blob/807eb8de6d7054cef1fac91cbf2e2a1eb37d30ed/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/processor/SimpleProcessLogger.java [3] https://github.com/apache/nifi/blob/31fba6b3332978ca2f6a1d693f6053d719fb9daa/nifi-api/src/main/java/org/apache/nifi/processor/AbstractProcessor.java [4] http://www.slf4j.org/legacy.html Andy LoPresto alopresto.apa...@gmail.com<mailto:alopresto.apa...@gmail.com> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 On Feb 10, 2016, at 7:54 PM, sudeep mishra <sudeepshekh...@gmail.com<mailto:sudeepshekh...@gmail.com>> wrote: Hi, What is the preferred practice for logging details for NiFi data flow? How can I use my own logging using log4j to log custom details for NiFi data flow. Thanks & Regards, Sudeep -- Thanks & Regards, Sudeep
