Hi Maryam, We have just finished a preliminary evaluation of Kakfa + collectd. On the Kafka side, we have simulated up to 700 servers sending telemetry data and have experimented with different broker setups with regards to number of partitions, replication factor, etc... What we see is average latency values with traffic around 8ms. However, the CDF has a long tail (out to 200ms) which have not tried to address yet.
We have prototyped an event-driven, ethernet interface link status plugin for collectd. The plugin produces no measurable load on the system and has a time from interrupt delivery to the plugin to a consumer on the Kafka bus of around 8-10ms. We would be interested in seeing your results as well. We are finishing up a document with the results. Aaron On Fri, Apr 21, 2017 at 8:04 AM, Tahhan, Maryam <[email protected]> wrote: > Hi folks > > > > I will be reducing the call next week to 30mins. > > > > [barometer] Update <April 18 2017> > > > > * Dev update: https://wiki.opnfv.org/display/fastpath/Barometer+ > Development+Updates > > > > * Test Update: > > * E release > > * Internally prepared scripts for IPMI functest – nearly complete > > * Mastered generating RPM under fuel. > > * Started preparing the Apex Setup. > > * No Apex Based RPM/packaging of barometer code: RPM package build > initiated à in progress: https://gerrit.opnfv.org/gerrit/#/c/32797/2 > > > > * E release planning, please provide your feedback 24th April. > > * Updated the release page with a proposal: https://wiki.opnfv.org/ > display/fastpath/Barometer+Release+Plan but still looking for > feedback/input > > * Action Maryam to add clock speed and power state (CPU + platform) to > the list of metrics for E. > > * Kafka to be added based on test results [ACTION] Maryam to follow up > with the team – might be beyond E release > > > > * Monitoring Agents Comparative study https://wiki.opnfv.org/ > display/fastpath/Monitoring+Agents+Comparative+Study > > * Completed answers to the question list – slides to be uploaded to the > wiki. > > > > * ETSI document TST008: > > * Barometer requirements for the metrics context reporting are now in > sync with TST008. > > * Approval request being put forward in May. > > > > * VES update > > * General VES code in the process of being refactored to python or Java > – to be determined by ONAP project. > > * Wiki updated for changes anticipated integration with DCAE and PNDA. > > > > * [TODO] run some tests on the host and the guest side to determine > accuracy of CPU metrics – In progress https://wiki.opnfv.org/ > display/fastpath/CPU+Utilisation+Case+Study > > * [*ACTION*] Maryam to update the wiki with results à *Done* > > * Results for a Test with a system (VM) under load à *Done* > > * Use 10 second averages à *Done* > > * create a wiki with the steps to reproduce à *Done* > > * use stress as the app in the VM à *Done* > > * [*ACTION*] Maryam to create a wiki with the setup steps à *Done*, > https://wiki.opnfv.org/display/fastpath/CPU+Utilisation+case+study+ > Configuration > > * [*ACTION*] Maryam to test with user time and system time > > * [*ACTION*] Maryam to test with synced time for the 1 and 10 second > interval à I think this can be done with cron > > * [*ACTION*] Maryam to check time sync between the host and the guest > > > > Kafka: > > Still investigating this > <https://specs.openstack.org/openstack/oslo-specs/specs/liberty/adding-kafka-support.html> > kafka > support in OpenStack to see what openstack projects and installers support > it. > > The next step is to install it and do some benchmarking with collectd and > kafka then collectd and rabbitmq… note some benchmarks are included in the > link above. à In progress > > > > Discussion topics wiki: > > https://wiki.opnfv.org/display/fastpath/Discussion+Topics > > > > All the best > > Maryam > > _______________________________________________ > opnfv-tech-discuss mailing list > [email protected] > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss > > -- AARON SMITH SENIOR PRINCIPAL SOFTWARE ENGINEER, NFVPE Red Hat <https://www.redhat.com/> 314 Littleton Rd, Westford, MA 01886 [email protected] M: 617.877.4814 <https://red.ht/sig>
_______________________________________________ opnfv-tech-discuss mailing list [email protected] https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
