Re: Nifi: how to split logs?
Thank you, this post is really helpful -- Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/
Re: Nifi: how to split logs?
Thank you , this post was really helpfull -- Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/
Re: Nifi: how to split logs?
Thank you, this post is really helpfull -- Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/
Re: Nifi: how to split logs?
Sally, Try this link ( https://community.hortonworks.com/articles/131320/using-partitionrecord-grokreaderjsonwriter-to-pars.html) . *GrokReader which helps to read log file with help of GrokExpression and in which you can parse by only error,info and warn data by using RouteOnAttribute.* Hope this helpful for you. On Mon, Nov 27, 2017 at 10:54 AM, sally wrote: > i want to parse only error ,info, warn data without provinance > > > > -- > Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/ >
Re: Nifi: how to split logs?
i want to parse only error ,info, warn data without provinance -- Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/
Re: Nifi: how to split logs?
Sally, What are you trying to parse out of nifi-app.log? If you are looking for provenance information, it may be easier and more straightforward to query provenance directly. On Fri, Nov 24, 2017 at 6:13 AM sally wrote: > I want to use nifi logs ( i mean nifi-app.log data) but when i try to get > this files usually i got loads of content on one error and i want to know > how can i filter all this data so that i could get only usefull information > > 1.should i change some configuration in logback.xml > 2.or i should use splitText ,splitContent or relative nifi processors > here is example of my log file data: > 2017-11-13 18:29:09,824 INFO [Provenance Maintenance Thread-3] > o.a.n.p.lucene.UpdateMinimumEventId Updated Minimum Event ID for Provenance > Event Repository - Minimum Event ID now 45299672017-11-13 18:29:09,824 INFO > [Provenance Maintenance Thread-3] o.a.n.p.PersistentProvenanceRepository > Successfully performed Expiration Action > org.apache.nifi.provenance.lucene.UpdateMinimumEventId@5c633b27 on > Provenance Event file ./provenance_repository/4529212.prov.gz in 3 > millis2017-11-13 18:29:09,824 INFO [Provenance Maintenance Thread-3] > o.a.n.p.lucene.DeleteIndexAction Removed expired Provenance Event file > ./provenance_repository/4529212.prov.gz2017-11-13 18:29:09,824 INFO > [Provenance Maintenance Thread-3] o.a.n.p.lucene.DeleteIndexAction Removed > expired Provenance Table-of-Contents file > ./provenance_repository/toc/4529212.toc2017-11-13 18:29:09,824 INFO > [Provenance Maintenance Thread-3] o.a.n.p.PersistentProvenanceRepository > Successfully performed Expiration Action > org.apache.nifi.provenance.expiration.FileRemovalAction@3f1fd7b on > Provenance Event file ./provenance_repository/4529212.prov.gz in 233296 > nanos2017-11-13 18:29:07,862 INFO [Provenance Repository Rollover Thread-1] > o.a.n.p.lucene.SimpleIndexManager Index Writer for > ./provenance_repository/index-1509974708000 has been returned to Index > Manager and is no longer in use. Closing Index Writer2017-11-13 > 18:29:07,911 > INFO [Provenance Repository Rollover Thread-1] > o.a.n.p.PersistentProvenanceRepository Successfully merged 16 journal files > (103463 records) into single Provenance Log File > ./provenance_repository/9476034.prov in 6719 milliseconds2017-11-13 > 18:29:07,911 INFO [Provenance Repository Rollover Thread-1] > o.a.n.p.PersistentProvenanceRepository Successfully Rolled over > > > > -- > Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/ >
Nifi: how to split logs?
I want to use nifi logs ( i mean nifi-app.log data) but when i try to get this files usually i got loads of content on one error and i want to know how can i filter all this data so that i could get only usefull information 1.should i change some configuration in logback.xml 2.or i should use splitText ,splitContent or relative nifi processors here is example of my log file data: 2017-11-13 18:29:09,824 INFO [Provenance Maintenance Thread-3] o.a.n.p.lucene.UpdateMinimumEventId Updated Minimum Event ID for Provenance Event Repository - Minimum Event ID now 45299672017-11-13 18:29:09,824 INFO [Provenance Maintenance Thread-3] o.a.n.p.PersistentProvenanceRepository Successfully performed Expiration Action org.apache.nifi.provenance.lucene.UpdateMinimumEventId@5c633b27 on Provenance Event file ./provenance_repository/4529212.prov.gz in 3 millis2017-11-13 18:29:09,824 INFO [Provenance Maintenance Thread-3] o.a.n.p.lucene.DeleteIndexAction Removed expired Provenance Event file ./provenance_repository/4529212.prov.gz2017-11-13 18:29:09,824 INFO [Provenance Maintenance Thread-3] o.a.n.p.lucene.DeleteIndexAction Removed expired Provenance Table-of-Contents file ./provenance_repository/toc/4529212.toc2017-11-13 18:29:09,824 INFO [Provenance Maintenance Thread-3] o.a.n.p.PersistentProvenanceRepository Successfully performed Expiration Action org.apache.nifi.provenance.expiration.FileRemovalAction@3f1fd7b on Provenance Event file ./provenance_repository/4529212.prov.gz in 233296 nanos2017-11-13 18:29:07,862 INFO [Provenance Repository Rollover Thread-1] o.a.n.p.lucene.SimpleIndexManager Index Writer for ./provenance_repository/index-1509974708000 has been returned to Index Manager and is no longer in use. Closing Index Writer2017-11-13 18:29:07,911 INFO [Provenance Repository Rollover Thread-1] o.a.n.p.PersistentProvenanceRepository Successfully merged 16 journal files (103463 records) into single Provenance Log File ./provenance_repository/9476034.prov in 6719 milliseconds2017-11-13 18:29:07,911 INFO [Provenance Repository Rollover Thread-1] o.a.n.p.PersistentProvenanceRepository Successfully Rolled over -- Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/
