Re: [vpp-dev] Streaming Telemetry with gNMI server

2019-02-03 Thread Jerome Tollet via Lists.Fd.Io
Hi Yohan & Stevan,
Great work. Thanks!
Jerome

Le 02/02/2019 23:35, « vpp-dev@lists.fd.io au nom de Yohan Pipereau » 
 a écrit :

Hi everyone,

Stevan and I have developed a small gRPC server to stream VPP metrics to
an analytic stack.

That's right, there is already a program to do this in VPP, it is
vpp_prometheus_export. Here are the main details/improvements regarding
our implementation:

* Our implementation is based on gNMI specification, a network standard
co-written by several network actors to allow configuration and
telemetry with RPCs.
* Thanks to gNMI protobuf file, messages are easier to parse and use a
binary format for better performances.
* We are using gRPC and Protobuf, so this is a HTTP2 server
* We are using a push model (or streaming) instead of a pull model. This
mean that clients subscribe to metric paths with a sample interval, and
our server streams counters according to the sample interval.
* As we said just before, contrary to vpp_prometheus_export, our
application let clients decide which metric will be streamed and how often.
* For interface related counters, we also provide conversion of
interface indexes into interface names.
Ex: /if/rx would be output as /if/rx/tap0/thread0
But at this stage, this conversion is expensive because it uses a loop
to collect vapi interface events. It is planned to write paths with
interface names in STAT shared memory segment to avoid this loop.

Here is the link to our project:
https://github.com/vpp-telemetry-pfe/gnmi-grpc

We have provided a docker scenario to illustrate our work. It can be
found in docker directory of the project. You can follow the guide named
guide.md.

Do not hesitate to give us feedbacks regarding the scenario or the code.

Yohan


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12151): https://lists.fd.io/g/vpp-dev/message/12151
Mute This Topic: https://lists.fd.io/mt/29635594/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-


Re: [vpp-dev] Streaming Telemetry with gNMI server

2019-02-02 Thread Ni, Hongjun
Hi Yohan and Stevan,

Thank you for your great work!

FD.io has a sub-project named Sweetcomb, which provides gNMI Northbound 
interface to upper application.
Sweetcomb project will push its first release on Feb 6, 2019.
Please take a look at below link for details from Pantheon Technologies:
https://www.youtube.com/watch?v=hTv6hFnyAhE 

Not sure if your work could be integrated with Sweetcomb project?

Thanks a lot,
Hongjun


-Original Message-
From: vpp-dev@lists.fd.io [mailto:vpp-dev@lists.fd.io] On Behalf Of Yohan 
Pipereau
Sent: Sunday, February 3, 2019 5:55 AM
To: vpp-dev@lists.fd.io
Cc: Stevan COROLLER 
Subject: [vpp-dev] Streaming Telemetry with gNMI server

Hi everyone,

Stevan and I have developed a small gRPC server to stream VPP metrics to an 
analytic stack.

That's right, there is already a program to do this in VPP, it is 
vpp_prometheus_export. Here are the main details/improvements regarding our 
implementation:

* Our implementation is based on gNMI specification, a network standard 
co-written by several network actors to allow configuration and telemetry with 
RPCs.
* Thanks to gNMI protobuf file, messages are easier to parse and use a binary 
format for better performances.
* We are using gRPC and Protobuf, so this is a HTTP2 server
* We are using a push model (or streaming) instead of a pull model. This mean 
that clients subscribe to metric paths with a sample interval, and our server 
streams counters according to the sample interval.
* As we said just before, contrary to vpp_prometheus_export, our application 
let clients decide which metric will be streamed and how often.
* For interface related counters, we also provide conversion of interface 
indexes into interface names.
Ex: /if/rx would be output as /if/rx/tap0/thread0 But at this stage, this 
conversion is expensive because it uses a loop to collect vapi interface 
events. It is planned to write paths with interface names in STAT shared memory 
segment to avoid this loop.

Here is the link to our project:
https://github.com/vpp-telemetry-pfe/gnmi-grpc

We have provided a docker scenario to illustrate our work. It can be found in 
docker directory of the project. You can follow the guide named guide.md.

Do not hesitate to give us feedbacks regarding the scenario or the code.

Yohan
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12141): https://lists.fd.io/g/vpp-dev/message/12141
Mute This Topic: https://lists.fd.io/mt/29635594/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-