On 12/02/2014 05:59 PM, Albert K wrote: > Hi, > > Yes, I did log the flows with the -F switch. What I need is for the > the data stored in the json field to be stored as individual fields > instead of current serialized in json format. It would enable me to > read and process the data without additional steps of de-serialization > the json data. Hope that it can be done.
So you do not seem to have used ES as it is doing exactly that. This said, the simplest solution is to write a simple JSON parser that dumps data to disk. ntopng is able to do that on the lua API, so you just need to code the lua script Luca > > Thanks. > > On Wed, Dec 3, 2014 at 12:51 AM, Luca Deri <[email protected]> wrote: >> On 12/02/2014 05:42 PM, Albert K wrote: >>> Hi, >>> >>> The storage format that I am looking for is for the data that was >>> stored in the sqlite table named flows with field named "json" to be >>> actually stored as individual fields (with easy identifiable field >>> names) . Hope this explain what i am looking for? Thank you. >> This is what -F with elasticsearch does. Did you try it? >> >> luca >>> On Tue, Dec 2, 2014 at 10:25 PM, Luca Deri <[email protected]> wrote: >>>> Hi Albert >>>> >>>> On 10/28/2014 09:57 AM, Albert K wrote: >>>>> Hi, >>>>> >>>>> I have a few questions regarding the SQLite Flow Dump database. I am >>>>> running ntopng v.1.2.2 (r8477) with -F db parameter >>>>> >>>>> 1) When I use "--json-labels" parameter there is no change in the >>>>> output of field "json" in the flow table. From the URL link below it >>>>> explain that the output should have decoded the key/fieldname instead >>>>> of numeric representation. Also when I looked in the source code of >>>>> perfs.cpp there is no parameter as per "--json-labels" Is it >>>>> deprecated or not implemented yet? >>>> --json-label is implemented on nProbe and not on ntopng. >>>> >>>>> http://www.ntop.org/ntop/introducing-ntopng-1-2/ >>>>> https://svn.ntop.org/svn/ntop/trunk/ntopng/doc/UserGuide.pdf >>>>> >>>>> 2) Is there a way to keep only certain number of days of data? For >>>>> example 10days round robin or round robin on a predetermine total >>>>> size. >>>> what is the format you have in mind for storing data? At the moment we >>>> support only counters or using -F you can dump data on SQlite or >>>> ElasticSearch >>>> >>>>> 3) The field "bytes", what does it represent? Is it the combined >>>>> total of received and sent? >>>> Where? >>>>> 4) What is the data does the content of the "json" field represent? >>>>> Can someone please provide me the decoded field names of the data? >>>> They are identified by symbolic labels defined in the netflow RFC 3954 >>>> and also supported by nProbe >>>> >>>> Regards Luca >>>> >>>>> Thanks. >>>>> _______________________________________________ >>>>> Ntop mailing list >>>>> [email protected] >>>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>> _______________________________________________ >>>> Ntop mailing list >>>> [email protected] >>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>> _______________________________________________ >>> Ntop mailing list >>> [email protected] >>> http://listgateway.unipi.it/mailman/listinfo/ntop >> _______________________________________________ >> Ntop mailing list >> [email protected] >> http://listgateway.unipi.it/mailman/listinfo/ntop > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop
