> From: Ciara Power [mailto:ciara.po...@intel.com] > Sent: Wednesday, April 8, 2020 6:50 PM > > This patchset extensively reworks the telemetry library adding new > functionality and simplifying much of the existing code, while > maintaining backward compatibility. > > This work is based on the previously sent RFC for a "process info" > library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 > However, rather than creating a new library, this patchset takes > that work and merges it into the existing telemetry library, as > mentioned above. > > The telemetry library as shipped in 19.11 is based upon the metrics > library and outputs all statistics based on that as a source. However, > this limits the telemetry output to only port-level statistics > information, rather than allowing it to be used as a general scheme for > telemetry information across all DPDK libraries. > > With this patchset applied, rather than the telemetry library being > responsible for pulling ethdev stats and pushing them into the metrics > library for retrieval later, each library e.g. ethdev, rawdev, and even > the metrics library itself (for backwards compatiblity) now handle > their > own stats. Any library or app can register a callback function with > telemetry, which will be called if requested by the client connected > via > the telemetry socket.
Great. Standardization across libraries is a good improvement. > The callback function in the library/app then > formats its stats, or other data, into a JSON string, and returns it to > telemetry to be sent to the client. I am strongly opposed to using JSON as the standard format in DPDK, and instead prefer a binary format with zero (or minimal) type conversion. Here is one reason why I dislike JSON for this: A part of our application samples 100k+ counters every second to be able to provide drill-down statistics through the GUI. Converting these counters from uint64_t to JSON and back to uint64_t for data processing is not going to fly. And I assume that we are not the only company developing equipment with drill-down statistics. I am aware that there is a difference between statistics for *drill-down and data processing* purposes and statistics for *telemetry eyeball viewing only* purposes, but the line is blurry, and I see a big risk of setting a path that leads to JSON being used in places where it shouldn't. Here is another reason why I dislike JSON for this: JSON is fine for the LAMP stack with REST protocols. But other systems use other protocols with other formats, e.g. the TICK stack uses an even simpler text based format. So DPDK based systems supporting the TICK stack will need to convert to first JSON format (in the DPDK libraries), and then from JSON format to InfluxDB format (in the DPDK application). I think that type conversion does not belong inside deep inside the DPDK libraries, but is a job for the DPDK application. However, DPDK could provide libraries for efficient bulk conversion to popular formats like JSON. And other formats, if they are relevant, e.g. ASN.1 used by old school SNMP.
