[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi all, Thanks for dialing in to the meeting this week. Going forward, in order to conserve everyone's time, we will be polling ahead of time for an agenda, and foregoing the online meeting in favor of voluntary email updates if no agenda is suggested. I will send out a reminder for an agenda the day before this scheduled meeting if one hasn't been suggested. Cheers, Brenden === Discussion === This meeting has become somewhat repetitive, and often lacks an agenda. The discussion that focuses around status isn't too useful. The goal of the meeting should be to discuss specific issues. Please send issues to discuss prior to the meeting, otherwise send regular updates (if useful) in email format. Brendan: Book is not visible online yet, waiting on publisher Reviewers are still looking at pre-copy-edit versions Question for kernel devs: how does the verifier handle divide by zero? Should the book call the BPF a VM or runtime? Answer: runtime It is also an instruction set. Any mention of turing completeness? ... waiting for bpf runtime written in bpf :) === Attendees === Brenden Blanco Michael Savisko Bjorn Topel Jesper Brouer Jakub Kicinski Daniel Borkmann Jiong Wang Alexei Starovoitov Joe Stringer Marco Leogrande Martin Lau Maciej Fijalkowski Brendan Gregg Dan Siemon John F Quentin Monnet Richard Elling -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1719): https://lists.iovisor.org/g/iovisor-dev/message/1719 Mute This Topic: https://lists.iovisor.org/mt/32046567/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi all, Thanks for joining in to the call today. As usual, here are my notes. Cheers, Brenden === Discussion === Daniel: - Some cilium debugging for loadbalancing case - Fix sendmsg hooks for unconnected udp - should fix dns resolution issue in cilium Yonghong: - Released new bcc version (0.10.0) - Some new revisions in LLVM for CO-RE support - bpf_send_signal is complete, support being added to libbpf Brendan: - nearing book deadline, send in your last minute reviews Alexei: - CFP for Linux Plumbers is open (Sep 9-11, Portugal) - https://www.linuxplumbersconf.org/event/4/abstracts/ - BPF subtrack will be taking place (some overlap with networking) Maciej: - working XSK (AF_XDP socket) support in libbpf - some issues without an obvious solution, details to be sent to mailing list === Attendees === Brenden Blanco Florian Lehner Paul Chaignon Dan Siemon Richard Elling Andy Gospodarek Marco Leogrande Maciej Fijalkowski Daniel Borkmann Alexei Starovoitov Jiong Wang Jakub Kicinski Jesper Brouer Brendan Gregg Mauricio Vasquez Joe Stringer -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1696): https://lists.iovisor.org/g/iovisor-dev/message/1696 Mute This Topic: https://lists.iovisor.org/mt/31842187/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[iovisor-dev] minutes: IO Visor/TSC Dev Meeting
Hi all, Thanks for joining the call today. As usual, here are my notes from the discussion. === Discussion === Alexei: LSF/MM - BPF track * overall went quite well * planning BPF conference for next year * notes @ http://vger.kernel.org/bpfconf2019.html Yonghong: * Compiler work to support compile-once-run-anywhere * LLVM code review fabricator - search for yonghong to find patch * Send signal bpf helper RFC1 has some issues * trying instead to send to current, even if in interrupt context * Other updates: some work for map of maps support in bcc John: * Some progress on bounded loop * Performing arithmetic on induction variables in one pass through loop * Many loops have simple enough induction variable strides to satisfy this simple algorithm * other sockmap and ktls fixes * release/acquire socket lock issue in ktls * fuzzing fixes Jiong: * 32 bit patchset work continues * Currently determining how to test * Setting up arm 32 bit qemu environment has issues * Cross-compiling kernel + selftest suite * Some offers to test on 32 bit x86 Brendan: * BPF book is currently being reviewed * If someone on the call would like to contribute reviews, reach out to Brendan * bpftrace syntax to be finalized * some compiler hints would be nice to pass __user pointer annotation to the scripts, otherwise explicit wrapper is necessary * necessary only for mixed mode tools Joe: * socket_assign helper for proxying support in bpf * tproxy working for tcp, working on udp * skc_lookup[_tcp] helper needed? * some leakiness in the socket api/bpf abstraction, needs to be worked out Daniel: * Out of bounds fix * cgroups hooks for sendmsg * breaks for DNS * add recvmsg hooks for reverse path, to avoid lookup failure * some minor fixes for connected (tcp) case Andrii: * Improving libbpf for BTF (and in prior kernels) * BTF to C converter * next: using BTF to define bpf maps === Attendees === Alexei Starovoitov Andrii Nakryiko Augusto Caringi Dan Siemon Jesper Brouer Richard Elling Brenden Blanco Jiong Yonghong Song Andy Gospodarek Daniel Borkmann Jakub Kicinski Brendan Gregg John Fastabend Mauricio Vasquez Paul Chaignon Joe Stringer -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1669): https://lists.iovisor.org/g/iovisor-dev/message/1669 Mute This Topic: https://lists.iovisor.org/mt/31633128/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for joining in to the call this week. As usual, here are my notes. Cheers, Brenden === Discussion === Yonghong: * Code reviews mostly * bpfd (client/server approach) effort paused * seeing if compile-once/run-everywhere can replace this effort * signal helper (from bpf prog to process) * hhvm accepts a signal, dumps stack, easier than collecting from bpf side Jiong: * Tuning 32 bit patch set * Power PC users did some testing and reported bugs in unit test * v5 expected soon Matheus: * Features and test improvements * Support for array index access * 'cat' feature * USDT Jesper: * Tried to play with BTF, but libbpf failed to load with some kernel configs * There are some known issues, Yonghong says that it will get some attention and hopefully improve soon. === Attendees === Brenden Blanco Matheus Marchini Dan Siemon Richard Elling Jesper Brouer Yonghong Song Jiong Wang Quentin Monnet -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1656): https://lists.iovisor.org/g/iovisor-dev/message/1656 Mute This Topic: https://lists.iovisor.org/mt/31451811/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for joining the call today. As usual, here are my notes. Thanks, Brenden === Discussion === Yonghong: - compile once run everywhere work continues - some prototypes for internal projects - some details to be presented at BPF microconference later this month - implement helper to send signals to other processes in the system - some bcc work on bpfd, patches to be reviewed - (bpfd is a remote bcc tool for low resource systems) Matheus: - macro support in bpftrace - porting to new version of llvm pass manager - old version has some memory issues - bpftrace issue #528 - request for jenkins support - Ubuntu 18.04 (4.15 or later) - ai for Brenden Daniel: - bpf global data support got merged - cilium - masquerade support native in bpf John: - bounded loop work has resumed in llvm, hopefully something to review in a couple weeks - some sockmap and ktls fixes Martin: - Socket local storage work - userspace looks like a regular map - a new helper to be implemented - local storage will be in the socket - memory charged to option memory of the socket Brendan: - Writing some advanced networking tools in bpftrace Dan: question about AF_XDP: - is the intention that user mem can be large? - 5-10k subscribers with ~10 packets each - may require some work to support hugepages - some of the docs on the tx use case are somewhat lacking === Attendees === Brenden Blanco Augusto Caringi John Fastabend Flavio Crisciani Michael Savisko Matheus Dan Siemon Marco Leogrande Yonghong Song Jakub Kicisnki Paul Chaignon Joe Stringer Daniel Borkmann Brendan Gregg Martin Lau Quentin Monnet Saeed -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1652): https://lists.iovisor.org/g/iovisor-dev/message/1652 Mute This Topic: https://lists.iovisor.org/mt/31217968/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Great, will have a look, thanks! Regards, Jiong On 05/04/2019 07:45, Y Song wrote: Hi, Jiong, To follow up the iovisor meeting discussion, the below is my prototype for an end_loop instruction in llvm: https://github.com/yonghong-song/llvm/commit/b83226772100317092cae6478229ed6ca3b9903c The goal is to help verifier to just focus on these marked cases, rejecting any other backedges. Please let me and John know if you get some better idea for bounded loop support in llvm/verifier. Thanks, Yonghong On Wed, Apr 3, 2019 at 11:49 AM Brenden Blanco wrote: Hi All, Thanks for the good discussion today! Below are my notes. Thanks, Brenden === Discussion === [...] Yonghong: * Compile once run anywhere work continues * bitfield handling bugs in IR/debuginfo Daniel: * Global support work continues * BTF side patches submitted to bpf mailing list * tests included Jiong: * 32 bit patch set * test methodology improvements * updated patches later in the week * some concerns around shifts, to be addressed in later improvements Andrii: * BTF and compile-once work integration to share prototype tool with Saeed Brendan: * Is there a tool to measure queue latency in qdisc->netdev layer? * debian/ubuntu are packaging bpftrace * except libbcc renamed to libbpf_cc * some issues with mixing iovisor's libbcc and debian's Jesper: * Fedora adding packaging support for libbpf Alexei: * systemd also adding support for libbpf - link to be provided? === Attendees === Brenden Blanco Andril Nakryiko Daniel Borkmann Jesper Brouer Jiong Wang Marco Leogrande Michael Savisko Paul Chaignon Quentin Monnet Alexei Starovoitov Saeed Flavio Rony Jonathan Lemon Brendan Gregg Dan Siemon Joe Stringer John -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1645): https://lists.iovisor.org/g/iovisor-dev/message/1645 Mute This Topic: https://lists.iovisor.org/mt/30527411/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi, Jiong, To follow up the iovisor meeting discussion, the below is my prototype for an end_loop instruction in llvm: https://github.com/yonghong-song/llvm/commit/b83226772100317092cae6478229ed6ca3b9903c The goal is to help verifier to just focus on these marked cases, rejecting any other backedges. Please let me and John know if you get some better idea for bounded loop support in llvm/verifier. Thanks, Yonghong On Wed, Apr 3, 2019 at 11:49 AM Brenden Blanco wrote: > > Hi All, > > Thanks for the good discussion today! Below are my notes. > > Thanks, > Brenden > > === Discussion === [...] > Yonghong: > * Compile once run anywhere work continues > * bitfield handling bugs in IR/debuginfo > > Daniel: > * Global support work continues > * BTF side patches submitted to bpf mailing list > * tests included > > Jiong: > * 32 bit patch set > * test methodology improvements > * updated patches later in the week > * some concerns around shifts, to be addressed in later improvements > > Andrii: > * BTF and compile-once work integration > to share prototype tool with Saeed > > Brendan: > * Is there a tool to measure queue latency in qdisc->netdev layer? > * debian/ubuntu are packaging bpftrace > * except libbcc renamed to libbpf_cc > * some issues with mixing iovisor's libbcc and debian's > > Jesper: > * Fedora adding packaging support for libbpf > > Alexei: > * systemd also adding support for libbpf - link to be provided? > > === Attendees === > Brenden Blanco > Andril Nakryiko > Daniel Borkmann > Jesper Brouer > Jiong Wang > Marco Leogrande > Michael Savisko > Paul Chaignon > Quentin Monnet > Alexei Starovoitov > Saeed > Flavio > Rony > Jonathan Lemon > Brendan Gregg > Dan Siemon > Joe Stringer > John > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1644): https://lists.iovisor.org/g/iovisor-dev/message/1644 Mute This Topic: https://lists.iovisor.org/mt/30527411/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All,
Thanks for the good discussion today! Below are my notes.
Thanks,
Brenden
=== Discussion ===
Michael:
* https://github.com/savisko/katran/tree/xdp_off
* Mellanox presentation on XDP + Katran
* offload tc XDP programs to hardware nic
* Example application: Katran from Facebook
* Control is implemented as a C++ library (example is open source)
* Katran DP already implemented in XDP
* Parsing + extract flow ID
* lookup key generation
* counter update
* packet modified to forward to other IP
* Accelerate marking of flows in hardware
* XDP metadata to pass mark field from hardware to xdp program
* struct xdp_md_mark {
__u32 mark)
};
* if (mark_ptr + 1 <= data)
markID = mark_ptr->mark;
* per-CPU XDP map to convert mark -> real flow information
mark == 0 implies new flow
* original XDP slow-path has get_packet_dst() to create LRU mapping
* modified version uses perf event output to notify acceleration helper to
install flow mark in hardware
* perf results:
100 flows: 40+% performance improvements
10k flows: 0-50% performance improvements depending on #rx queues used
Software: 25Mpps
Hardware: 37-39Mpps
* considering changing implementation to mark real server instead of flow
id, to reduce number of entries kept in L1 cache
Yonghong:
* Compile once run anywhere work continues
* bitfield handling bugs in IR/debuginfo
Daniel:
* Global support work continues
* BTF side patches submitted to bpf mailing list
* tests included
Jiong:
* 32 bit patch set
* test methodology improvements
* updated patches later in the week
* some concerns around shifts, to be addressed in later improvements
Andrii:
* BTF and compile-once work integration
to share prototype tool with Saeed
Brendan:
* Is there a tool to measure queue latency in qdisc->netdev layer?
* debian/ubuntu are packaging bpftrace
* except libbcc renamed to libbpf_cc
* some issues with mixing iovisor's libbcc and debian's
Jesper:
* Fedora adding packaging support for libbpf
Alexei:
* systemd also adding support for libbpf - link to be provided?
=== Attendees ===
Brenden Blanco
Andril Nakryiko
Daniel Borkmann
Jesper Brouer
Jiong Wang
Marco Leogrande
Michael Savisko
Paul Chaignon
Quentin Monnet
Alexei Starovoitov
Saeed
Flavio
Rony
Jonathan Lemon
Brendan Gregg
Dan Siemon
Joe Stringer
John
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#1640): https://lists.iovisor.org/g/iovisor-dev/message/1640
Mute This Topic: https://lists.iovisor.org/mt/30527411/21656
Group Owner: iovisor-dev+ow...@lists.iovisor.org
Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [iovisor-dev] minutes: IO Visor TSC/Dev Meeting
On Tue, Apr 2, 2019 at 10:48 AM Saeed Mahameed wrote: > > Hi Brenden, > > I am sending this on behalf of Michael Savisko, he is having some > difficulties sending emails to the iovisor list. Sending to the iovisor-dev mailer is gated by a signup requirement, in order to reduce spam. The signup process should be pretty painless, I believe it just requires going through an email validation step: https://lists.iovisor.org/g/iovisor-dev/join > > Michael is working on real world use cases for XDP acceleration. > He would like to present and discuss his work and analysis on > accelerating Katran load balancer [1] via meta data offloads. > He will need 10 minutes and will share some slides, i hope we can push > this to tomorrow's meeting agenda. Sounds good! > > Katran modified code is on Github: > https://github.com/savisko/katran/tree/xdp_off > > [1] > https://code.fb.com/open-source/open-sourcing-katran-a-scalable-network-load-balancer/ > Thanks, > Saeed. > > On Wed, Mar 20, 2019 at 3:32 PM Brenden Blanco wrote: > > > > Hi all, > > > > Thanks for joining the discussion today. Here are the notes; however, this > > was > > a longer discussion and I'm sure I missed some things. > > > > Cheers, > > Brenden > > > > === Discussion === > > > > Yonghong: > > * Some internal BTF work > > * Compiler support for static variables > > * Some compile-once-run-everywhere work > > * Looking for help with issues regarding libbpf packaging/dependencies > > * Issue to continue offline, not concluded on the call > > * Issue related to function->function call in bcc and compiler optimizations > > * Jiong offers to debug the codegen using 32 bit mode > > > > Saeed: > > * XDP driver statistics standardization > > * All drivers run same entry point for xdp progs > > * Why not account stats here? > > * Even though xdp program can implement its own statistics > > * Many drivers are already paying stats accounting cost > > * Just remove unused stats from driver? > > * Stats may be used in debugging, but FB for instance is guarding with > > static key, wouldn't want extra stats on by default > > * Allocating resources for tx queue/redirect? > > * Is there a better way to allocate resources when it isn't known that a > > program will need queues > > * One approach is to attach dummy bpf program > > * Resource allocation point when configuring devmap? > > * Seems like a clean enough solution, doesn't solve all cases but moves > > the > > ball forward > > * BTF metadata structure registration > > * Should be queryable from userspace, don't yet have an API for that > > * Netlink vs syscall? > > * No silver bullet for all use cases > > * Hesitation for creating a new object to describe existing objects (bpf > > progs, maps) > > * BTF is metadata conceptually different from maps, progs > > * ethtool? unlikely due to lack of code ownership > > * For buffers, something like devlink is more appropriate > > * For BTF, bpf() syscall works > > * BTF for statistics description (ethtool replacement?) > > > > Daniel: > > * verification of static data is working, patches coming soon > > > > === Attendees === > > Brenden Blanco > > Michael Savisko > > Alexei Starovoitov > > Daniel Borkmann > > Jakub Kicinski > > Neerav Parikh > > Paul Chaignon > > Saeed > > Marco Leogrande > > Jiong Wang > > Andrii Nakryiko > > Yonghong Song > > William Tu > > Joe Stringer > > John > > Maciej Fijalkowski > > Martin Lau > > Mauricio Vasquez > > Piotr Raczynski > > Quillian Rutherford > > > > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1637): https://lists.iovisor.org/g/iovisor-dev/message/1637 Mute This Topic: https://lists.iovisor.org/mt/30527411/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi Brenden, I am sending this on behalf of Michael Savisko, he is having some difficulties sending emails to the iovisor list. Michael is working on real world use cases for XDP acceleration. He would like to present and discuss his work and analysis on accelerating Katran load balancer [1] via meta data offloads. He will need 10 minutes and will share some slides, i hope we can push this to tomorrow's meeting agenda. Katran modified code is on Github: https://github.com/savisko/katran/tree/xdp_off [1] https://code.fb.com/open-source/open-sourcing-katran-a-scalable-network-load-balancer/ Thanks, Saeed. On Wed, Mar 20, 2019 at 3:32 PM Brenden Blanco wrote: > > Hi all, > > Thanks for joining the discussion today. Here are the notes; however, this was > a longer discussion and I'm sure I missed some things. > > Cheers, > Brenden > > === Discussion === > > Yonghong: > * Some internal BTF work > * Compiler support for static variables > * Some compile-once-run-everywhere work > * Looking for help with issues regarding libbpf packaging/dependencies > * Issue to continue offline, not concluded on the call > * Issue related to function->function call in bcc and compiler optimizations > * Jiong offers to debug the codegen using 32 bit mode > > Saeed: > * XDP driver statistics standardization > * All drivers run same entry point for xdp progs > * Why not account stats here? > * Even though xdp program can implement its own statistics > * Many drivers are already paying stats accounting cost > * Just remove unused stats from driver? > * Stats may be used in debugging, but FB for instance is guarding with > static key, wouldn't want extra stats on by default > * Allocating resources for tx queue/redirect? > * Is there a better way to allocate resources when it isn't known that a > program will need queues > * One approach is to attach dummy bpf program > * Resource allocation point when configuring devmap? > * Seems like a clean enough solution, doesn't solve all cases but moves > the > ball forward > * BTF metadata structure registration > * Should be queryable from userspace, don't yet have an API for that > * Netlink vs syscall? > * No silver bullet for all use cases > * Hesitation for creating a new object to describe existing objects (bpf > progs, maps) > * BTF is metadata conceptually different from maps, progs > * ethtool? unlikely due to lack of code ownership > * For buffers, something like devlink is more appropriate > * For BTF, bpf() syscall works > * BTF for statistics description (ethtool replacement?) > > Daniel: > * verification of static data is working, patches coming soon > > === Attendees === > Brenden Blanco > Michael Savisko > Alexei Starovoitov > Daniel Borkmann > Jakub Kicinski > Neerav Parikh > Paul Chaignon > Saeed > Marco Leogrande > Jiong Wang > Andrii Nakryiko > Yonghong Song > William Tu > Joe Stringer > John > Maciej Fijalkowski > Martin Lau > Mauricio Vasquez > Piotr Raczynski > Quillian Rutherford > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1636): https://lists.iovisor.org/g/iovisor-dev/message/1636 Mute This Topic: https://lists.iovisor.org/mt/30527411/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi all, Thanks for joining the discussion today. Here are the notes; however, this was a longer discussion and I'm sure I missed some things. Cheers, Brenden === Discussion === Yonghong: * Some internal BTF work * Compiler support for static variables * Some compile-once-run-everywhere work * Looking for help with issues regarding libbpf packaging/dependencies * Issue to continue offline, not concluded on the call * Issue related to function->function call in bcc and compiler optimizations * Jiong offers to debug the codegen using 32 bit mode Saeed: * XDP driver statistics standardization * All drivers run same entry point for xdp progs * Why not account stats here? * Even though xdp program can implement its own statistics * Many drivers are already paying stats accounting cost * Just remove unused stats from driver? * Stats may be used in debugging, but FB for instance is guarding with static key, wouldn't want extra stats on by default * Allocating resources for tx queue/redirect? * Is there a better way to allocate resources when it isn't known that a program will need queues * One approach is to attach dummy bpf program * Resource allocation point when configuring devmap? * Seems like a clean enough solution, doesn't solve all cases but moves the ball forward * BTF metadata structure registration * Should be queryable from userspace, don't yet have an API for that * Netlink vs syscall? * No silver bullet for all use cases * Hesitation for creating a new object to describe existing objects (bpf progs, maps) * BTF is metadata conceptually different from maps, progs * ethtool? unlikely due to lack of code ownership * For buffers, something like devlink is more appropriate * For BTF, bpf() syscall works * BTF for statistics description (ethtool replacement?) Daniel: * verification of static data is working, patches coming soon === Attendees === Brenden Blanco Michael Savisko Alexei Starovoitov Daniel Borkmann Jakub Kicinski Neerav Parikh Paul Chaignon Saeed Marco Leogrande Jiong Wang Andrii Nakryiko Yonghong Song William Tu Joe Stringer John Maciej Fijalkowski Martin Lau Mauricio Vasquez Piotr Raczynski Quillian Rutherford -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1624): https://lists.iovisor.org/g/iovisor-dev/message/1624 Mute This Topic: https://lists.iovisor.org/mt/30527411/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi Guys, My agenda for next meeting: 1) unifying and centralizing XDP statistics accounting [1]. 2) XDP resource management, User API [2]. 3) XDP meta data via btf (in kernel BTF registration). 4) all of the above issues share one common problem, which is the lack of a unified user interface without it, We really can't make a real progress. I just sent a proposal [3] for away to achieve the unified interface, please look it up and let me know your thoughts. [1] https://github.com/xdp-project/xdp-project/blob/master/xdp-project.org#statistics-per-xdp-action [2] https://github.com/xdp-project/xdp-project/blob/master/xdp-project.org#better-ndo_xdp_xmit-resource-management [3] Subject: "[RFC][Proposal] BPF Control MAP" Thanks, Saeed On Wed, Mar 6, 2019 at 12:51 PM Brenden Blanco wrote: > > Hi All, > > Thank you for joining the call today. Here are my notes from the discussion. > > Thanks, > Brenden > > === Discussion === > Brenden: > * Plan to tag release to coincide with kernel 5.0 > > Brendan: > * Speaking this weekend at SCaLE in Los Angeles > > Yonghong: > * LLVM work > * compile once - run anywhere WIP > * support for static variables > > Daniel: > * Global data support work in kernel continues > * Ability to lock maps as read-only > * bugfixes after merge window > > Alexei: > * Some thoughts on future work of BPF > * especially with introduction of BTF > * overall needs concerted effort to improve debuggability > * BTF for programs itself with source/type/layout information > * structures for maps and global data > * suggest to always require type information >(already turned on by default in bcc and supported by llvm) > * Some extra hoops to jump through for driver embedded BPF > * to be enabled with a sysctl > * kernel support is ready > * some long tail of support - e.g. systemd has raw assembly BPF > * kconfig option - eventual deprecation > * if kernel is default strict, llvm should automatically emit BTF as well > * memcg accounting patch status? > * Daniel - still being worked on > * proposal to enable the same accounting for verifier memory > * helps to enable verifier multithreading > > Jakub: > * question regarding global data atomicity > * Daniel - requires read once / write once instructions to work properly > * some todo work on documentation, interpreter + jit implementations > * depends on architecture (machine word size guarantees only) > > Jesper: > * which llvm release supports BTF > * landed in December - will be in 8.0, better in 9.0 > * working on tutorial for xdp at netdev > * https://www.netdevconf.org/0x13/session.html?tutorial-XDP-hands-on > * soliciting feedback > * https://github.com/xdp-project/xdp-tutorial/ > > Saeed: > * request to devote some time in the next meeting to iron out some XDP issues > * please send an agend in reply to the reminder email before next call > * prepare discussion over email in between time > > === Attendees === > Alexei Starovoitov > Marco Leogrande > Mauricio Vasquez > Paul Chaignon > Brenden Blanco > Jiong Wang > Yonghong Song > Daniel Borkmann > Jesper Brouer > Quentin Monnet > Dan Siemon > Jakub Kicinski > Saeed > John > Yutaro > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1604): https://lists.iovisor.org/g/iovisor-dev/message/1604 Mute This Topic: https://lists.iovisor.org/mt/30289974/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thank you for joining the call today. Here are my notes from the discussion. Thanks, Brenden === Discussion === Brenden: * Plan to tag release to coincide with kernel 5.0 Brendan: * Speaking this weekend at SCaLE in Los Angeles Yonghong: * LLVM work * compile once - run anywhere WIP * support for static variables Daniel: * Global data support work in kernel continues * Ability to lock maps as read-only * bugfixes after merge window Alexei: * Some thoughts on future work of BPF * especially with introduction of BTF * overall needs concerted effort to improve debuggability * BTF for programs itself with source/type/layout information * structures for maps and global data * suggest to always require type information (already turned on by default in bcc and supported by llvm) * Some extra hoops to jump through for driver embedded BPF * to be enabled with a sysctl * kernel support is ready * some long tail of support - e.g. systemd has raw assembly BPF * kconfig option - eventual deprecation * if kernel is default strict, llvm should automatically emit BTF as well * memcg accounting patch status? * Daniel - still being worked on * proposal to enable the same accounting for verifier memory * helps to enable verifier multithreading Jakub: * question regarding global data atomicity * Daniel - requires read once / write once instructions to work properly * some todo work on documentation, interpreter + jit implementations * depends on architecture (machine word size guarantees only) Jesper: * which llvm release supports BTF * landed in December - will be in 8.0, better in 9.0 * working on tutorial for xdp at netdev * https://www.netdevconf.org/0x13/session.html?tutorial-XDP-hands-on * soliciting feedback * https://github.com/xdp-project/xdp-tutorial/ Saeed: * request to devote some time in the next meeting to iron out some XDP issues * please send an agend in reply to the reminder email before next call * prepare discussion over email in between time === Attendees === Alexei Starovoitov Marco Leogrande Mauricio Vasquez Paul Chaignon Brenden Blanco Jiong Wang Yonghong Song Daniel Borkmann Jesper Brouer Quentin Monnet Dan Siemon Jakub Kicinski Saeed John Yutaro -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1598): https://lists.iovisor.org/g/iovisor-dev/message/1598 Mute This Topic: https://lists.iovisor.org/mt/30289974/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for dialing in to the meeting today. Here are my notes from the discussion. Thanks, Brenden === Discussion === Alexei: - Discussion with some kernel devs regarding BPF memory model - some documentation to be done - (currently avoiding uapi enforcement on content of bpf program itself) - introduce standard header file for bpf programs to include - add read_once, write_once primitive - llvm will understand this - 64 bit jit must implement these instructions LSFMM will have a BPF track - CFP closes this Friday, send your proposals if you have them - Or send an email if you simply wish to attend Saeed: - LPC discussion followup - metadata parsing in hardware - way to notify/raise events userspace of new flow marks detected and trigger injection of tc rule? - use perf ring buffer, available in kernel samples/ Brendan: - bpftrace function tracing use case in java - needs couple fixes - uprobe to be able to attach heap locations instead of inode-only - tell c2 compiler to pause moving functions around - looking for ways to get past libc/libpthread backtrace walking (usually compiled without frame pointer) - hoping for Intel LBR support, but not available in many cloud VMs yet === Attendees === Brenden Blanco Andrii Nakryiko Dan Siemon Marco Leogrande Jesper Brouer Jerome Marchand Tom Stellard Jakub Kicinski Bjorn Topel Martin Lau Andy Gospodarek Paul Chaignon Jiong Wang Alexei Starovoitov Daniel Borkmann Saeed Quentin Monnet Mauricio Vasquez Matheus Marchini Joe Stringer Florian Brendan Gregg -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1582): https://lists.iovisor.org/g/iovisor-dev/message/1582 Mute This Topic: https://lists.iovisor.org/mt/29954587/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thank you for joining us on the call today. As usual, here are the notes. -Brenden === Discussion === Yonghong: - working on libbpf features - working on btf support in bcc - need to use upstream libbpf apis, in process of upstreaming/converting - llvm support is working (9.0) Joe: - cilium: getting static data to be updated from elf section - allows iproute2 to fetch 32bit values from data section - populate instructions that refer to the data at load time - different data sizes not yet supported/tested - would like to incorporate global data support on the kernel side Martin: - bpf socket and tc socket patches posted - what happens with sockets in timewait state? - is there a usecase for such sockets? not at this moment, but still under discussion Discussion about allocating sockopt storage for bpf use cases John: - Discussion about ipsec integration with bpf - bpf program to compute hash key instead of setting up ipsec rules - currently in idea phase only Daniel: - cilium ipvlan support nearing completion Brendan: - bpftrace integration/improvements making nice progress === Attendees === Brenden Blanco Yonghong Song Marco Leogrande Jesper Brouer Joe Stringer Mauricio Vasquez Quentin Monnet Daniel Borkmann Jiong Wang Martin Lau Brendan Gregg John Fastabend -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1562): https://lists.iovisor.org/g/iovisor-dev/message/1562 Mute This Topic: https://lists.iovisor.org/mt/29685292/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [iovisor-dev] minutes: IO Visor TSC/Dev Meeting
On 24/01/2019 02:33, Brenden Blanco wrote: Jiong: * respin jmp32 patch set - v2 - implementation in working state - isa for subregister looking good - codegen giving smaller results (num of instructions) - will resend patch set after fixing test failure Finished re-spin and rebased on top of latest bpf-next. My set will potentially conflict with the following three other bpf-next pending patches/sets on selftests Makefile and test_verifier.c: - Stanislav's LLVM_READELF change on Makefile. - Alexei's spin_lock change on test_verifier.c. - Jakub's test_verifier.c refactor on test_verifier.c. The conflict should be easy to resolve. For test_verifier.c, my set just adds new tests at the end of the test array. I will send out v3 shortly. Regards, Jiong -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1558): https://lists.iovisor.org/g/iovisor-dev/message/1558 Mute This Topic: https://lists.iovisor.org/mt/29521656/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for dialing in to the call today. Here are notes from the discussion. Thanks, Brenden === Discussion === Alexei: * Spinlock patches being worked on * Proposal to standardize the BPF ELF format/api to that used by libbpf/bpftool - First user might be iproute2 conversion - Should support multi-language bindings (go) - Approved Brenden: * Tagged bcc release Yonghong: * bringing libbpf into bcc - few related bugs * bringing BTF support into bcc - WIP - issue on getting remapped file (from compilation stage) info into bcc - getting btf type during map creation in right order in bcc...needs some thought * build id support added - first user in trace.py Jiong: * respin jmp32 patch set - v2 - implementation in working state - isa for subregister looking good - codegen giving smaller results (num of instructions) - will resend patch set after fixing test failure John: * patches to be sent out for sockmap and ktls * the parser in the filter on the ktls rx side needs some work - there are some edge cases between ktls and other parsers Jakub: * refactoring test cases for verifier improvements Daniel: * Helping to refactor/review test verifier code - splitting up tests by category * Cilium work to add in second datapath (ipvlan based) === Attendees === Brenden Blanco Alexei Starovoitov Jakub Kicinksi Martin Lau Mauricio Vasquez Quentin Monnet Dan Siemon Daniel Borkmann Jesper Brouer Andy Gospodarek Maciej Fijalkowski Yonghong Song John F Paul Chaignon -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1557): https://lists.iovisor.org/g/iovisor-dev/message/1557 Mute This Topic: https://lists.iovisor.org/mt/29521656/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [iovisor-dev] minutes: IO Visor TSC/Dev Meeting
On Wed, Jan 9, 2019 at 11:59 AM Brenden Blanco wrote: > > Hi All, > > Thanks for joining the call today after a short holiday hiatus. Below are my > notes from the call. > > === Discussion === > Brenden > * Adopted new project kubectl-trace into github.com/iovisor from the author's > (@fntlnz) personal github > - Adds support for running bpftrace as a 'kubectl trace' command in a > kubernetes environment > - Presented at Kubecon 2018: > > https://kccna18.sched.com/event/GrYw/ebpf-powered-distributed-kubernetes-performance-analysis-lorenzo-fontana-influxdata > - Please give it a spin and contribute/comment on > github.com/iovisor/kubectl-trace > * Nice work Lorenzo! > > Yonghong > * funcinfo and line info merged BTF support merged into kernel > * clang debug info for bpf > - use -g from clang to generate such info that gets passed along to BTF > - some blocking issues to bring this feature into bcc > - btf info not being handled properly by dynamic linker > * bcc plan to release by end of the week to coincide with kernel release > * wip: add 128 bit int support > > Brendan > * Compiler errors with linux 5.0-rc (asm goto) > * Working on documentation and packaging for bpftrace > - lots of small bugs to fix marching towards 1.0 > > Alexei > * Proposing dates and format for BPF/IO Visor specific technical miniconf in > the > springtime > - March-May timeframe > - 2 days? > > William > * Adding AF_XDP support to vSphere driver Sorry, I mean AF_XDP for veth driver. Thank you William > > > === Attendees === > Brenden Blanco > Quentin Monnet > Jakub Kicinski > Alexei Starovoitov > Jiong Wang > Maciej Fijalkowski > William Tu > Yonghong Song > Brendan Gregg > Daniel Borkmann > Martin Lau > Neerav Parikh > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1555): https://lists.iovisor.org/g/iovisor-dev/message/1555 Mute This Topic: https://lists.iovisor.org/mt/28988043/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for joining the call today after a short holiday hiatus. Below are my notes from the call. === Discussion === Brenden * Adopted new project kubectl-trace into github.com/iovisor from the author's (@fntlnz) personal github - Adds support for running bpftrace as a 'kubectl trace' command in a kubernetes environment - Presented at Kubecon 2018: https://kccna18.sched.com/event/GrYw/ebpf-powered-distributed-kubernetes-performance-analysis-lorenzo-fontana-influxdata - Please give it a spin and contribute/comment on github.com/iovisor/kubectl-trace * Nice work Lorenzo! Yonghong * funcinfo and line info merged BTF support merged into kernel * clang debug info for bpf - use -g from clang to generate such info that gets passed along to BTF - some blocking issues to bring this feature into bcc - btf info not being handled properly by dynamic linker * bcc plan to release by end of the week to coincide with kernel release * wip: add 128 bit int support Brendan * Compiler errors with linux 5.0-rc (asm goto) * Working on documentation and packaging for bpftrace - lots of small bugs to fix marching towards 1.0 Alexei * Proposing dates and format for BPF/IO Visor specific technical miniconf in the springtime - March-May timeframe - 2 days? William * Adding AF_XDP support to vSphere driver === Attendees === Brenden Blanco Quentin Monnet Jakub Kicinski Alexei Starovoitov Jiong Wang Maciej Fijalkowski William Tu Yonghong Song Brendan Gregg Daniel Borkmann Martin Lau Neerav Parikh -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1554): https://lists.iovisor.org/g/iovisor-dev/message/1554 Mute This Topic: https://lists.iovisor.org/mt/28988043/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Thanks all for joining in to the call today. As usual, here are my notes. === Discussion === Jiong: Continuing work on jmp32 - some work on verifier side - needs to work on test support - new test mode with alu 32 option - LLVM in alu 32 mode caused some verifier rejections...investigating Yonghong: Some BTF related bug cleanup Working on LLVM -> BCC -> bpf() functionality prototype - Some dependencies on LLVM/clang patch integration...may cause slowdown Planning to integrate libbpf into libbcc - More updates to be sent to mailing list in next couple weeks William: Continuing OVS/AF_XDP work - Presenting at OVS conf in San Jose - Some further use cases to be presented (tunneling, virtual port) Jakub: Attempted to package bpftool in debian, but ran into gplv3 libbfd linking issue - Will attempt to dual-license and ask permission from contributors, check email - Some concerns about size of library Jesper: Page pool patches are in a working state - Working through prototype code on an arm64 board - Some early integration with the mlnx driver that uses page pools as well ** For version-controlled collaboration on XDP related topics, join Jesper and others using the https://github.com/xdp-project/xdp-project repo John: sockmap/ktls work improvements to enable Cilium use cases Joe: looking into feedback on patches related to signed values === Attendees === Alexei Starovoitov Andy Gospodarek Augusto Caringi David Beckett Jakub Kicinski Jesper Brouer John Fastabend Yonghong Song Paul Chaignon Paul Jardetzky Quentin Monnet William Tu Yifeng Sun Neerav Parikh Brenden Blanco -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1532): https://lists.iovisor.org/g/iovisor-dev/message/1532 Mute This Topic: https://lists.iovisor.org/mt/28479922/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for joining in to the discussion today. Below are my notes from the call. Hope everyone is looking forward to good discussions at the LPC BPF track in Vancouver in two weeks! Since that overlaps with the usual IO Visor call, we will have our next meeting in 4 weeks time instead. Thanks, Brenden === Discussion === Brenden: * tracking some buildbot failures * will look at adding fedora 29 target * to tag 0.8.0 soon Yonghong: * updated kernel headers * working on llvm internals for BTF support Alexei: * for those presenting at LPC/bpf track, please send rough draft of slides by tomorrow * significant effort to do deduplication of dwarf->BTF of vmlinux symbols: 300MB->2MB * feature: atomic spin lock/unlock callable from bpf programs * 1 lock only * no helper calls within critical section * BTF will be mandatory for such programs * more to be presented at plumbers Daniel: * Some security fixes * LPC slides John: * Work on paper/slides for LPC * Sockmap support is enabled in Cilium! * (mellanox?) working to add upstream support in openssl for kTLS + sockmap Jakub: * working on getting libbpf packaged separately and released by distros * FB has external mirror github.com/libbpf/libbpf === Attendees === Brenden Blanco Paul Chaignon Dan Siemon Nic Viljoen Alexei Starovoitov Andy Gospodarek Jakub Kicinski Daniel Borkmann Yonghong Song Mauricio Vasquez JohnF Joe Stringer Quentin Monnet William Tu Quillian Rutherford -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1514): https://lists.iovisor.org/g/iovisor-dev/message/1514 Mute This Topic: https://lists.iovisor.org/mt/27810162/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for dialing in today. As usual, here are my notes. === Discussion === Alexei * bpf-next closing this weekend, to remain closed during merge window - get your last minute fixes in * schedule for bpf microconference is near-finalized, availabe on the website Daniel * ktls and sockmap changes have been merged - few remaining fixes being worked on * some perf and libbpf improvements John * ktls and sockmap work as well - currently adding support for it into cilium - integrating into CI infrastructure - adding metadata push support similar to xdp approach Jiong * more work on 32 bit support in verifier - identified a couple regressions, fixing those - how to scale verifier implementation to large numbers of instructions? - will work on this William * looking at xdp support in vmxnet3 driver Brendan * bpftrace launched! - github.com/iovisor/bpftrace - new issues being found, fixed - recently fixed compilation for llvm7 * being used to find real issues in netflix - fd leak detector * some kernel feature requests identified: - ppid - access to rusage statistics Mauricio * Will split out patch to just include stackmap, other pieces still being discussed Joe * writing tests for sockmap lookup patches - possible to have socket lookup from xdp? === Attendees === Brenden Blanco Alexei Daniel Borkmann Jakub Kicinski Joe Stringer Nic Viljoen Paul Chaignon Quentin Monnet Jiong Wang Yifeng Sun Andy Gospodarek Brendan Gregg John F William Tu Mauricio Vasquez Martin Lau -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1502): https://lists.iovisor.org/g/iovisor-dev/message/1502 Mute This Topic: https://lists.iovisor.org/mt/27388869/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [iovisor-dev] minutes: IO Visor TSC/Dev Meeting
On Wed, Oct 3, 2018 at 11:26 AM Brenden Blanco wrote: > > Hi all, > > Thanks for joining the call this week. Here are my notes of the items > discussed. > > -Brenden > > === Discussion === > > Yonghong: > - BTF support in LLVM WIP > - prototype is approaching functionality > - to post to llvm mailing list soon (enabled through -g -target=bpf mode) The link to the llvm patch which generates .BTF and .BTF.ext sections: https://reviews.llvm.org/D52950 > > William: > - OVS + AF_XDP performance improvements > - more batching on tx side > - add more asynchronous logic > - goes from 5Mpps -> 14Mpps forwarding > - tested ixgbe AF_XDP v2 support - seems to be working > > Daniel: > - kTLS and sockmap integration WIP with John > - planning to post this week > - bpf plumbers track submissions is closed > > JohnF: > - Working on Cilium pieces for kTLS+sockmap > > Joe: > - Socket lookup helpers were merged this week > > Jesper: > - working on XDP paper with John and Daniel > - Gave talk in Paris @Kernel Recipes > - > http://people.netfilter.org/hawk/presentations/KernelRecipes2018/XDP_Kernel_Recipes_2018.pdf > > Mauricio: > - Working to address review comments this week on bpf queue and stackmap > > === Attendees === > Brenden Blanco > Yonghong Song > Jakub Kicinski > Mauricio Vasquez > JohnF > Daniel Borkmann > Joe Stringer > William Tu > Jesper Brouer > Jiong Wang > Nic Viljoen > Quentin Monnet > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1492): https://lists.iovisor.org/g/iovisor-dev/message/1492 Mute This Topic: https://lists.iovisor.org/mt/26721077/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi all, Thanks for joining the call this week. Here are my notes of the items discussed. -Brenden === Discussion === Yonghong: - BTF support in LLVM WIP - prototype is approaching functionality - to post to llvm mailing list soon (enabled through -g -target=bpf mode) William: - OVS + AF_XDP performance improvements - more batching on tx side - add more asynchronous logic - goes from 5Mpps -> 14Mpps forwarding - tested ixgbe AF_XDP v2 support - seems to be working Daniel: - kTLS and sockmap integration WIP with John - planning to post this week - bpf plumbers track submissions is closed JohnF: - Working on Cilium pieces for kTLS+sockmap Joe: - Socket lookup helpers were merged this week Jesper: - working on XDP paper with John and Daniel - Gave talk in Paris @Kernel Recipes - http://people.netfilter.org/hawk/presentations/KernelRecipes2018/XDP_Kernel_Recipes_2018.pdf Mauricio: - Working to address review comments this week on bpf queue and stackmap === Attendees === Brenden Blanco Yonghong Song Jakub Kicinski Mauricio Vasquez JohnF Daniel Borkmann Joe Stringer William Tu Jesper Brouer Jiong Wang Nic Viljoen Quentin Monnet -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1490): https://lists.iovisor.org/g/iovisor-dev/message/1490 Mute This Topic: https://lists.iovisor.org/mt/26721077/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi, All, Below is the today's meeting notes. Thanks for joining the meeting. Please amend if you see any inaccurate or missing information. Yonghong Discussion Yonghong: - worked and upstreamed bpftool net - worked on BTF for function type and LLVM for BTF. The goal is to have LLVM directly generating BTF to make it extensible (e.g., including debug line info etc.). William: - Tested AF_XDP on intel/NFP nic card. Using dpdk to generate traffic. Baseline rx: 17Mpps, L2 forwarding: 15Mpps. With OVS: 12Mpps, L2 forwarding: 3.7Mpps, potential implementation overhead with OVS - the test without intel meltdown configs to minimize perf impact. Mauricio: - queue/stack map discussion on netdev. Need to sort out some API issues Joe: - Good progress with socket lookup support and ref tracking in the bpf programs Jiong: - local data flow optimizer as a mini library could be used by nfp or other drivers or by verifier itself - pass info to jit to generate better code Attendees Yonghong Song Brendan Gregg Daniel Borkmann Jesper Brouer Jiong Wang Joe Stringer Mauricio Vasquez Quentin Monnet William Tu -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1484): https://lists.iovisor.org/g/iovisor-dev/message/1484 Mute This Topic: https://lists.iovisor.org/mt/25781882/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for joining the call today. Here are my notes! Cheers, Brenden === Discussion === Brenden: * 0.7.0 tag released to support 4.18 * Flaky test on fc26 - usdt3 - monitor and perhaps relax contraints in test Yonghong: * Discussing API for BTF support in kernel * prototyping currently the kernel interface * will need a few more weeks * Introspection improvements * Duplicating some ip/tc functionality into bpftool * Reason: want one integrated tool for all bpf introspection * Some debate about what the value of duplicating code is * Need to avoid too many tools * Lots of non-net users: cgroups, tc, xdp, perf * dual license libbpf to converge the code? Alexei: * bpf-next opened, swamped with code reviews Jiong: * 32 bit support * standalone flow control analyzer in progress * will try to remove extra config flags William: * AF_XDP for OVS * without zero copy driver, perf tests * > 3Mpps rx so far * working on port->port forwarding, with memcpy * needs zero-copy driver to improve tx speeds * will try with i40e driver === Attendees === Brenden Blanco Jakub Kicinski Jiong Wang Yonghong Song Jesper Brouer Alexei Starovoitov Joe Stringer Brendan Gregg William Tu JohnF -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1474): https://lists.iovisor.org/g/iovisor-dev/message/1474 Mute This Topic: https://lists.iovisor.org/mt/25212981/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Thanks all for joining the call today. As usual, here are my notes. -Brenden === Discussion === Yonghong: * Internal discussions for BTF related work * adding information to jit instructions * debug annotations * Introspection for networking * Device walker, qdisc for finding xdp/bpf programs * Tag release after .increment bug is fixed Alexei: * Trying to improve introspection of bpf * Add LLVM class for emitting BTF debug info * Possible standalone LLVM tool for converting dwarf->btf * pahole fixes coming soon * plan to use this tool for annotating kernel build to avoid heavy LLVM dependency William: * AFXDP patch sent last week for OVS * Performance still lagging due to extra memcpy * Next iteration will try zerocopy mode * Despite kasan, still seeing death due to oom-killer * Intel folks helping with debug Daniel: * Fixes for tcp ulp, sockmap, xdp redirect with tailcalls * Work continues on sockmap, KTLS * CFP is open for BPF track at Plumbers! John: * KTLS and sockmap work * BPF in KTLS working, cleaning up patches for submission * Bounded loop work possibly to be discussed at LPC Bjorn: * AFXDP zerocopy patches to be submitted when bpf-next opens * ixgbe/i40e patches to be included, or separately through intel tree === Attendees === Brenden Blanco Quentin Monnet Sandipan Daniel Borkmann Josh Kuba Kicinski William Tu Jiong Wang Alexei Yifeng Sun Yonghong Song Bjorn Topel Brendan Gregg John F Mauricio Vasquez -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1462): https://lists.iovisor.org/g/iovisor-dev/message/1462 Mute This Topic: https://lists.iovisor.org/mt/24923308/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for dialing in today. As usual, here are my notes. Cheers! -Brenden === Discussion === Brenden: * f28 and ubuntu18.04 support * 0.6.1 tagged with those packages * will try to get python3 testing enabled Yonghong: * Catching up with some mailing list questions * Going to try using BTF for profiling with FB internal tools * encode source code information for bpf program into btf * some introspection enhancements Jiong: * Progress on bpf llvm backend support for memcpy * patch set is out for review * working on better 32 bit support, llvm+verifier * 32 bit load/store into stack * needs better data structure in verifier for tracking <8B writes PJ: via email: """ Right now we're working on converting the work we did on HW hint providing into the BTF format, like we discussed in the last meeting. We're making decent progress, and should be able to share something soon to see what people think. We're also looking at how to connect the format specification with a communication mechanism to the drivers to program them to provide the metadata. Both of these are ongoing. """ Daniel: * Setting up list for plumbers schedule, lots of talks already submitted === Attendees === Brenden Blanco Marco Leogrande Jiong wang Flavio Crisciani Kuba Kicinski Mauricio Vasquez Yonghong Song Alexander Duyck Joe Stringer Martin Lau Saeed Andy Gospodarek Daniel Borkmann -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1401): https://lists.iovisor.org/g/iovisor-dev/message/1401 Mute This Topic: https://lists.iovisor.org/mt/23816913/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, As usual, here are my notes from the call today. -Brenden === Discussion === Brenden: - Packaging changes - have Ubuntu 18.04 buildbot tested and ready - Working on merging Fedora 28 changes with review comments William: ovs-ebpf rfc - stack size limitation - hoping to use pointer to map to save stack space instead of copying - discussing decision tree data structure for megaflow lookup - how to do conntrack, defrag? - trying a native bpf map implementation for conntrack - defrag may need a helper, looking at cilium implementation for help - af_xdp experiments Daniel: - bugfixes from syzkaller and other reports - KTLS and bpf integration - wip Saeed: - xdp metadata - ongoing mailing list discussions - BTF approach - still investigating - con: requires recompile for new hardware - hoping to standardize on metadata format amongst vendors - so far, sounds like couple vendors are working with sharing rfcs Brendan: - Nadav from VMware presented hyper-upcalls concept with ebpf support at conference === Attendees === Brenden Blanco Daniel Borkmann William Tu Yifeng Saeed Jakub Kicinski Aasif Shaikh Brendan Gregg Martin Lau PF Waskiewicz Uri Joe Stringer -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1365): https://lists.iovisor.org/g/iovisor-dev/message/1365 Mute This Topic: https://lists.iovisor.org/mt/23250735/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for attending the meeting today. As usual, here are my notes. === Discussion === Brenden: - working on fc28 packaging - pr available - includes some python3 fixes - not to be included in next release tag :( - v0.6.0 tagged Yonghong: - Some rewriter changes, working with Paul - 4.17 changed some syscall parameters - some fixes in bcc, when __syscall prefix is seen - tracepoint structure bugfix - introduce option to change license option Alexei: - bcc release schedule - want to release within short timeframe of kernel release - have tried to keep backwards compatible, but still highly dependent on kernel Saeed: - xdp offloads update - working on examples for using the new api/metadata - perf numbers to be published after that - currently experimenting with dynamic hw metadata - e.g.: vlan, csum complete, hash, flow tag - can hardware provide offsets to tcp data as a new metadata field? Okash: - BTF - some WIP patches for pretty printing maps, currently for array maps John: - sockhash fixes from fuzzbot - bugs in locking and ipv6 ports - Working on RFC for bounded loop support - need to expand from finding loop induction vars to all induction vars - llvm likes to build nested loops, and always jumps with a conditional in the middle, makes finding loop induction variables hard - also important to add overflow checks - possible to modify llvm to make verifying easier? - annotations? how to rely on them for safety... - introduce rep counter - more crazy ideas discussed - pass SSA, dom tree info to verifier Brendan: - bcc+btf support? - not yet, still working on dwarf->btf converter - pahole has some gaps to be able to convert kernel dwarf - probably good enough though === Attendees === Brenden Blanco Opher Bjorn Topel Marco Leogrande Martin Lau Nic Viljoen Saeed Jakub Kicinski George Jesper Brouer Alexander Duyck Quentin Monnet Daniel Borkmann William Tu Yonghong Song Brendan Gregg Yifeng Sun John Fastabend Okash -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1335): https://lists.iovisor.org/g/iovisor-dev/message/1335 Mute This Topic: https://lists.iovisor.org/mt/22182008/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi all, Thanks for joining in the discussion today. As usual, here are my notes. -Brenden === Discussion === Brenden: - Adding fedora 28 buildbot/packages - debugging some test failures - Tag release 0.7.0 after that Yonghong: - Some more python3 fixes - Some rewriter changes, breakages - Tool to do introspection for kprobe, uprobe John: - Working for certain classes of loops - some caveats with searching for loop variable declarations - only considers first order induction variables Bjorn: - some concerns about descriptor format - target to update next week - to disable support in the meantime - kconfig/uapi header removal? - return not supported for socket options in meantime - support for brcm, mlnx next merge window Saeed: - for AF_XDP, starting work on xdp offload for metadata/flow-mark concept - when loading xdp program, request specific offloads - e.g. vlan stripping, modifies headroom calculation - flow hash in metadata, avoid caching packet before forwarding w/ cpuhash - to come up with a proposal next week === Attendees === Brenden Blanco Alexei Starovoitov Andy Gospodarek Bjorn Topel Brendan Gregg Jesper Brouer Magnus Yonghong Song Quentin Monnet Saeed Kuba Kicinski Alexander Duyck JohnF William Tu ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
Re: [iovisor-dev] minutes: IO Visor TSC/Dev Meeting
On 17/05/2018 05:38, Brenden Blanco via iovisor-dev wrote: Hi All, Thanks for attending the meeting today. As usual, here are my notes. === Discussion === Bjorn - AF_XDP zero copy implementation RFC on netdev - some uapi concerns, may need to back out changes - but, over next couple weeks before merge window, discuss and improve in hopes of merging - clean up non-intel code, respin, leave out i40e patches at the end - separate i40e cleanup spin - bcm may try to join as 2nd vendor to flesh out uapi - some discussion of common code organization - xdp in separate file helps backporting Yonghong - some python3 fixes - proposals for improved container support - rewriter complexity increasing - tracing event introspection Martin - BTF id landed - working on bpftool support - still using pahole to convert dwarf - still unclear how integration with llvm will look Jiong - dominator tree info RFCs pushed - significant mem/exec time costs to run (e.g. 2x more, 3x slower) Just for the record, if I listened correctly, John proposed to introduce a switch for the new CFG infrastructure, i.e, we keep the exisiting DFS based check_cfg, the new CFG code will only be enabled when check_cfg detected a loop and wants know whether it is bounded, by this we could avoid extra mem/exec time cost on programs that don't have loop. Regards, Jiong John - induction variable tracking in progress - next: min/max value tracking both: to work on topic branch under bpf-next William - tried workaround from Yonghong - still need to dig into root cause Joe - implementation of sock lookup going through iterations - concerns over locking in bpf being a leaky implementation detail === Attendees === Brenden Blanco Alexei Starovoitov Bjorn Topel Daniel Borkmann Jesper Brouer Jiong Wang Magnus Karlsson Quentin Monnet Joe Stringer Andy Gospodarek Ferris Brendan Gregg Jakub Kicinski William Tu Yonghong Song Alexander Duyck Sandipan John F ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for attending the meeting today. As usual, here are my notes. === Discussion === Bjorn - AF_XDP zero copy implementation RFC on netdev - some uapi concerns, may need to back out changes - but, over next couple weeks before merge window, discuss and improve in hopes of merging - clean up non-intel code, respin, leave out i40e patches at the end - separate i40e cleanup spin - bcm may try to join as 2nd vendor to flesh out uapi - some discussion of common code organization - xdp in separate file helps backporting Yonghong - some python3 fixes - proposals for improved container support - rewriter complexity increasing - tracing event introspection Martin - BTF id landed - working on bpftool support - still using pahole to convert dwarf - still unclear how integration with llvm will look Jiong - dominator tree info RFCs pushed - significant mem/exec time costs to run (e.g. 2x more, 3x slower) John - induction variable tracking in progress - next: min/max value tracking both: to work on topic branch under bpf-next William - tried workaround from Yonghong - still need to dig into root cause Joe - implementation of sock lookup going through iterations - concerns over locking in bpf being a leaky implementation detail === Attendees === Brenden Blanco Alexei Starovoitov Bjorn Topel Daniel Borkmann Jesper Brouer Jiong Wang Magnus Karlsson Quentin Monnet Joe Stringer Andy Gospodarek Ferris Brendan Gregg Jakub Kicinski William Tu Yonghong Song Alexander Duyck Sandipan John F ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
Re: [iovisor-dev] minutes: IO Visor TSC / Dev Meeting
2018-05-03 1:39 GMT+02:00 Brenden Blanco via iovisor-dev : > Hi All, > > Thanks for joining the discussion today. As usual, here are my notes. > > -Brenden > > === Discussion === > > John: > - sockmap fixes pushed > - fixed broken tests > - sockmap hash impl next > - bounded loop rfc making progress > - cilium code using sockhash almost finished as well > > Yonghong: > - python3 test cleanup > - except memleak tool > - raw tracepoint support in bcc > - runqlat tool > - bio tracing > - more precise than kprobes > - next work: introspection - bpf trace program in perf > > Martin: > - BTF id patch WIP > - BTF id -> BTF map > - next: support function pointer type in BTF > > William: > - moving bpf tunnel example -> selftest > - hitting issue with long running test tunnel cases > - try enabling kasan to debug > - verifier issue bugfix (ctx + constant arithmetic) WIP > > Daniel: > - finalizing removal of some inst from jit > - few bugfixes in x86 jit > > Alexei: > - AF_XDP patches being reviewed, looks ok > Unfortunately, I missed the call. Appreciate that AF_XDP was took into discussion! Thanks for the review! > Jesper: > - Testing AF_XDP patches currently > - no driver changes needed with xdp_redirect > - driver changes still needed for zerocopy > - intel: 12mpps, mlnx: 9mpps drop > Again, thanks a lot for spending time on the AF_XDP patchset! Cheers, Björn > Brendan: > - CFP for USENIX LISA in October, Nashville TN > > === Attendees === > Brenden Blanco > Alex Reece > Jakub Kicinski > Martin Lau > Quentin Monnet > Sandipan > Jiong Wang > William Tu > Alexei Starovoitov > Alexander Duyck > John Fastabend > Jesper Brouer > Daniel Borkmann > Yonghong Song > Brendan Gregg > Ferris > ___ > iovisor-dev mailing list > iovisor-dev@lists.iovisor.org > https://lists.iovisor.org/mailman/listinfo/iovisor-dev ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
Re: [iovisor-dev] minutes: IO Visor TSC / Dev Meeting
On Wed, May 2, 2018 at 4:39 PM, Brenden Blanco via iovisor-dev wrote: > Hi All, > > Thanks for joining the discussion today. As usual, here are my notes. > > -Brenden > > === Discussion === > > John: > - sockmap fixes pushed > - fixed broken tests > - sockmap hash impl next > - bounded loop rfc making progress > - cilium code using sockhash almost finished as well > > Yonghong: > - python3 test cleanup > - except memleak tool This is the test_tools_memleak issue I mentioned. https://github.com/iovisor/bcc/issues/1685 I would be great if somebody who knows python (esp. python3) better can help take a look. ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC / Dev Meeting
Hi All, Thanks for joining the discussion today. As usual, here are my notes. -Brenden === Discussion === John: - sockmap fixes pushed - fixed broken tests - sockmap hash impl next - bounded loop rfc making progress - cilium code using sockhash almost finished as well Yonghong: - python3 test cleanup - except memleak tool - raw tracepoint support in bcc - runqlat tool - bio tracing - more precise than kprobes - next work: introspection - bpf trace program in perf Martin: - BTF id patch WIP - BTF id -> BTF map - next: support function pointer type in BTF William: - moving bpf tunnel example -> selftest - hitting issue with long running test tunnel cases - try enabling kasan to debug - verifier issue bugfix (ctx + constant arithmetic) WIP Daniel: - finalizing removal of some inst from jit - few bugfixes in x86 jit Alexei: - AF_XDP patches being reviewed, looks ok Jesper: - Testing AF_XDP patches currently - no driver changes needed with xdp_redirect - driver changes still needed for zerocopy - intel: 12mpps, mlnx: 9mpps drop Brendan: - CFP for USENIX LISA in October, Nashville TN === Attendees === Brenden Blanco Alex Reece Jakub Kicinski Martin Lau Quentin Monnet Sandipan Jiong Wang William Tu Alexei Starovoitov Alexander Duyck John Fastabend Jesper Brouer Daniel Borkmann Yonghong Song Brendan Gregg Ferris ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO VIsor TSC/Dev Meeting
Hi All, Thanks for attending the meeting today. Here are my notes from the call. Cheers, Brenden === Status === Yonghong - Debugging incorrect user stack - fastpath vs slowpath register difference - should be updated in 4.9+ (LTS flavors) - user stack id - doesn't have full access, trying to improve - bpf perf event introspection Daniel cgroup bind and connect patches merged - 3 new hooks for container management - used by fb sockmap ingress support added raw tracepoint - use all tp arguments directly from bpf some prep patches for upcoming features - btf, few others clang compiled kernel issue with bpffs - c lang constant merging flag turned off Joe sock lookup progress - improvements to verifier for tracking reference scope - next rfc soon John Some patches merged, some just missed this window - next support for lookup by hash Adding Cilium support for sockmap soon Rearranging selftests/examples for sockmap Jakub adding support for offloading of a few features - inline map helpers - atomic add - some performance improvements Next window - perf ring output support Working on 32 bit register support in llvm Martin BTF support V2 - pretty print from kernel - includes pahole converter next: LLVM directly generate BTF with -g === Attendees === Dan Siemon Brenden Blanco Alex Reece Alexander Duyck Daniel Borkmann Jakub Kicinski Joe Stringer Mauricio Vasquez Yonghong Song Martin Quentin Monnet Tom George Wilson Andy Gospodarek Jiong Wang Saeed Francois Joel jcanseco ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi all, Thanks for joining the discussion today, as usual my notes of the discussion are included below. Cheers, Brenden === Discussion === Alexei - microkernel/umh work - see lkml Yonghong - working on skb segmentation problem - ipv4/v6 transition - bpf on bpf clsact ingress Brenden - bcc release? - planning to go with a 1.0 tag soon Ferris - rust library for interfacing with bpf - more details to be announced in the future Jesper - Pushing internally to get bpf support officially enabled (rhel8) - some progress detected on af_xdp patches John - sockmap sendmsg/sendfile merged - ingress redirect upcoming - sockmap hashmap lookup support - fixes for mem charge leak - reorganize samples into selftests - documentation work Brendan - some minor tools update to use tracepoints - some tools written again bpftrace Martin - CTF patches almost ready, to be posted in a couple weeks === Attendees === Brenden Blanco Andy Gospodarek Daniel Borkmann Ferris Logan Blyth Tom Alexei Joe Stringer Jesper Brouer Jakub Kicinski George Wilson Yonghong Song Martin Lau Quentin Monnet Edwin Peer Brendan Gregg ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi all, Thanks for joining the call today! Please see my notes from the discussion included below. === Status === Sargun: seccomp-ebpf - verifier hurdles - wants synchronous filtering as minimum feature to allow the verifier risk - current rcu locking approach was nacked discussion about how bpf punts to userspace daemon - problems with passing metadata back and forth Jesper: xdp redirect memory return api rfc - patches under review on ml - some questions about dma mapping in the api - how to unmap page when device has already gone down? - or when page pool is full? Issue: enabling redirect to other device with no xdp rx program running currently results in silent failure. - Have an ethtool flag to enable tx devices -> reconfigure queues? Brendan: bpftrace - higher level frontend to bcc/bpf - making good progress - 70% feature complete - symbol resolution missing - dereference of c structs - depends on ctf? - test suite exists - uses llvm-ir diff against previous builds to sanity check - packaging? - tbd === Attendees === Brenden Blanco Quentin Monnet Tom Alexander Duyck Jiong Wang Sandipan Sargun Dhillon Jakub Kicinski Saeed Daniel Borkmann Ferris Jesper Brouer Brendan Gregg ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi all, Thanks for attending the call this week. My notes are included below. === Updates === DTrace GPLed in Dec 2017 - To help with Oracle Linux port - Should we take dtrace kernel code into linux? - opinion: probably not necessary now, more applicable 5 years ago - How about dtrace frontend? - may have more short term value - combine with bpftrace? - who would work on it? - userspace tools are now UPL - how does that work with apache2? - CTF - currently forking the format into BTF - exceed 32k types, other limitations - still, is the code useful to combine? bpfilter announcement - lots of lwn/reddit/hackernews comments Brenden: - Fixes for python3 string encoding - please test tools with python3 -W and submit PRs/issues - todo: push clang ir rewriter to github branch Yonghong: - Kernel related bugfixes - memory leak in shared lpm map Daniel: - bpfilter work - retpoline work in tail call code of x86 jit - also arm64 John: - some more sockmap changes - tx hooks - more testing, cleanup - optimizations in followup patches Joe: - Soliciting feedback for socket sharing patches - looking into storing data along with socket - scratch accessible from bpf program - How to protect scratch area? - new lock helper idea Sargun: - Asking for feedback on seccomp ebpf changes - desires access to read-only map for - use case for scratch buffer primitive as well Brendan: - New pcp-bcc-pmda plugin by redhat - tbd: integrate into vector Jesper: - XDP redirect work - bugfixes - page free and return API next - getting help from mellanox devs - XDP bulk xmit for rx === Attendees === Brenden Blanco Eduard Serra Alexei Starovoitov Joe String Jakub Kicinski Jiong Wang Mauricio Vasquez Quentin Monnet Saeed Sandipan Sargun Dhillon Brendan Gregg Yonghong Song Jesper Brouer Alexander Duyck Edwin Peer PJ Waskiewicz Daniel Borkmann Nic Viljoen John Fastabend Alexander Duyck ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
Re: [iovisor-dev] minutes: IO Visor TSC/Dev Meeting
On Wed, Feb 21, 2018 at 11:55 AM, Brenden Blanco wrote: > Hi all, > > Thanks for attending the call this week. My notes are included below. > > === Updates === > > DTrace GPLed in Dec 2017 > - To help with Oracle Linux port > - Should we take dtrace kernel code into linux? > - opinion: probably not necessary now, more applicable 5 years ago > - How about dtrace frontend? > - may have more short term value > - combine with bpftrace? > - who would work on it? > - userspace tools are now UPL - how does that work with apache2? > - CTF > - currently forking the format into BTF > - exceed 32k types, other limitations > - still, is the code useful to combine? > > bpfilter announcement > - lots of lwn/reddit/hackernews comments > > Brenden: > - Fixes for python3 string encoding > - please test tools with python3 -W and submit PRs/issues > - todo: push clang ir rewriter to github branch As requested, I have pushed the WIP code to https://github.com/iovisor/bcc/tree/ir-probe-rewrite. Only one of the two test cases currently works. The basic idea is to use the pointer alias info to track which loads read from on-stack memory and maps, and treat the rest as unsafe reads and rewrite them into bpf_probe_read calls. Map data is not yet handled. > > Yonghong: > - Kernel related bugfixes > - memory leak in shared lpm map > > Daniel: > - bpfilter work > - retpoline work in tail call code of x86 jit > - also arm64 > > John: > - some more sockmap changes > - tx hooks > - more testing, cleanup > - optimizations in followup patches > > Joe: > - Soliciting feedback for socket sharing patches > - looking into storing data along with socket > - scratch accessible from bpf program > - How to protect scratch area? > - new lock helper idea > > Sargun: > - Asking for feedback on seccomp ebpf changes > - desires access to read-only map for > - use case for scratch buffer primitive as well > > Brendan: > - New pcp-bcc-pmda plugin by redhat > - tbd: integrate into vector > > Jesper: > - XDP redirect work > - bugfixes > - page free and return API next > - getting help from mellanox devs > - XDP bulk xmit for rx > > === Attendees === > Brenden Blanco > Eduard Serra > Alexei Starovoitov > Joe String > Jakub Kicinski > Jiong Wang > Mauricio Vasquez > Quentin Monnet > Saeed > Sandipan > Sargun Dhillon > Brendan Gregg > Yonghong Song > Jesper Brouer > Alexander Duyck > Edwin Peer > PJ Waskiewicz > Daniel Borkmann > Nic Viljoen > John Fastabend > Alexander Duyck ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for joining the call today. As usual, here are my notes. === Status === Yonghong: - Bug fixes - Python 2/3 becoming default - Tools need some fixes - Brenden has some patches in a branch to address this JohnF: - Fixes for sockmap - socket cleanup wasn't working correctly - AF_XDP socket work Daniel: - Bug fixes - X86 jit fixes Alexei: - new feature work: Connecting to raw tracepoint, w/ arguments Jesper: - Merged suricata bpf support to upstream - Expected to work with 4.16 - some fixes incoming for bpf tree - Some work on AF_XDP planned - splitting up patches Joe: - Socket lookup from bpf - native socket pointer type in bpf - reference counting passed back to bpf program - verifier confirms alloc==free - more info soon === Attendees === Andy Gospodarek Brenden Blanco Jakub Kicinski Alexei Starovoitov Logan Blyth Marco Leogrande Quentin Monnet Daniel Borkmann Jiong Wang Ram Kumar Edwin Peer Jesper Brouer Yonghong Song Mauricio Vasquez John Fastabend PJ Waskiewicz Joe Stringer Alexander Duyck Brendan Gregg ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for attending the meeting today. As usual, here are my notes. -Brenden === Status === Alexei: - Closing last minute fixes for 4.15 - expect stable fixes to follow Yonghong: - Add some trace examples - addressing recurring questions about how to use generic tools - Fix trace pipe issue (4.13) - task struct layout clang.h confusion - Ask to Brenden: share LLVM pass rewriter in a branch Martin: - CTF (rename to BTF?) - can do file format validation - working pretty printer in kernel - missing unit tests - plan to add it to github somewhere - SO_REUSEPORT + bpf - add socket array - based on sockmap idea - john: have a receive hook without stream parser, might be useful Alban: - FOSDEM in first weekend of Feb Daniel: - Fixes for 4.15 - BPF exceptions Jesper: - Issue with libbpf loader for suricata - exception sections in elf files from llvm - Close to end of merge window John: - Respin sockmap changes - Fix for issue in test code where memory is leaked - Running a rx hook without the stream parser - removes heavyweight requirement for some usecases === Attendees === Brenden Blanco Alban Crequy Daniel Borkmann Mauricio Vasquez Marco Leogrande Alexander Duyck Alexei Starovoitov Jiong Wang Jesper Brouer Jakub Kicinski Yonghong Song John Fastabend Martin Lefau ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
Re: [iovisor-dev] minutes: IO Visor TSC/Dev Meeting
>From a glance on the BPF patches that deal with the speculative execution issues, I do not see any regard to direct packet access. I would presume a malicious user might access memory beyond PTR_TO_PACKET_END and leak the databy exploiting the speculative execution. Am I wrong in my understanding? Is it one of the patches that is in progress? Thanks, Nadav Brenden Blanco via iovisor-dev wrote: > Hi All, > > Thanks for attending today. Below you will find my notes from the call. > > Cheers, > Brenden > > === Status === > > Alexei: > - Security updates for BPF > - Patching uses of BPF for speculative execution > - variant 1, one more patch in progress > - variant 2, option to toggle off interpreter > > Yonghong: > - Fixes for bcc > - kernel fixes for get next key > - stack map iteration > - goal: keep consistent with hash map iteration > - issues for usdt, probe name overlap > > Brenden: > - Some work on CTF generation > > Daniel: > - Working on spectre patches and bugfixes > > JohnF: > - sockmap tx hook > - monitor sendmsg/sendpage > - sample improvements to ease development/debugging/benchmarking > > Jesper: > - xdp rx queue merged > - tracking followup fixes > - need to extend for memory return api > - todo: cpumap for generic xdp > - add cpumap support into suricata > - xdp_data_meta - want to combine skb->mark+cpumap to fine tune multicore > handling of packets > > > === Attendees === > Brenden Blanco > Alexander Duyck > Daniel Borkmann > Edwin Peer > Jakub Kicinski > Jesper Brouer > Jiong Wang > Quentin Monnet > Sandipan > Marco Leogrande > Mauricio Vasquez > Nic Viljoen > Andy Gospodarek > Maciej Fijalkowski > Yonghong Song > Rony Efraim > Alexei Starovoitov > Gordon Marler > John Fastabend > ___ > iovisor-dev mailing list > iovisor-dev@lists.iovisor.org > https://lists.iovisor.org/mailman/listinfo/iovisor-dev ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for attending today. Below you will find my notes from the call. Cheers, Brenden === Status === Alexei: - Security updates for BPF - Patching uses of BPF for speculative execution - variant 1, one more patch in progress - variant 2, option to toggle off interpreter Yonghong: - Fixes for bcc - kernel fixes for get next key - stack map iteration - goal: keep consistent with hash map iteration - issues for usdt, probe name overlap Brenden: - Some work on CTF generation Daniel: - Working on spectre patches and bugfixes JohnF: - sockmap tx hook - monitor sendmsg/sendpage - sample improvements to ease development/debugging/benchmarking Jesper: - xdp rx queue merged - tracking followup fixes - need to extend for memory return api - todo: cpumap for generic xdp - add cpumap support into suricata - xdp_data_meta - want to combine skb->mark+cpumap to fine tune multicore handling of packets === Attendees === Brenden Blanco Alexander Duyck Daniel Borkmann Edwin Peer Jakub Kicinski Jesper Brouer Jiong Wang Quentin Monnet Sandipan Marco Leogrande Mauricio Vasquez Nic Viljoen Andy Gospodarek Maciej Fijalkowski Yonghong Song Rony Efraim Alexei Starovoitov Gordon Marler John Fastabend ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Thanks all for attending today. Below you may find the notes from the call. We will be skipping the next call and resuming on Jan 10. See you all in the new year! === Status === Yonghong: - bcc: Fixed struct with alignment/int128 quirks - match in python - kernel: query prog array - used for event hooks with multiple programs attached - debugging use case - kernel: uprobe speedup patch landed Jesper: - submitted Quentin: - working on bpf syscall man page / helpers man page - where should this work go? - A: should go in kernel.org somewhere git.kernel.org (this includes kernel, libc, etc.) if automatically generated, need to occassionally sync from kernel to manpage repo sync once per release - contact Michael Kerisk Jiong: - Continuing 32bit bpf support in clang - support for rotate instruction? more optimal than shift combinations - benchmark with jhash inlined use cases - send experimental patches to yonghong/daniel Daniel: - Some ptregs fixes into bpf tree - Adding development documentation - Fixes for some jits - Work continues on bpf qdisc John: - Porting sockmap for other use case (ULP layer) - Reviewing call patches from Alexei Rony: - Q: Progress on BPF offload? - A: not much discussion yet - What does a BPF (offloaded) -> BPF (XDP) interface look like? - some vendors may accept P4 -> microcode (through firmware api) that can as fallback be compiled into BPF - should we avoid an out of band api for offloading the parsing program into silicon? - John has tried such APIs in the past, and rejected due to not wanting to add new api besides bpf+tc === Attendees === Brenden Blanco Jakub Kicinski Jiong Wang Quentin Monnet Rony Efraim Mauricio Vasquez Andy Gospodarek Alexei Starovoitov Yonghong Song Daniel Borkmann John Fastabend Brendan Gregg Nic Viljoen Edwin Peer ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for attending the call this week, as usual here are the notes of the discussions and updates. === Netdev === LWN summary article for Netconf+Netdev is available: https://lwn.net/Articles/738912/ Discussions: - XDP cpumaps - XDP offload - How to offload xdp metadata hints? - c-like language to describe hw - llvm -> set of instrinsics to program hardware - Sockmap - TCP hooks to modify tcp behavior? - mss window clamping - performance data - How to do page recycling across drivers? - some issues with ixgbe put_page approach - ndo op to unlink pages - New AF_PACKET version for fast userspace - to be renamed - possible XDP/other BPF hook to forward pages into userspace === Status === Yonghong - uprobe perf improvements - emulate push instruction - 50% improvement - verifier improvements - bpf_probe_read constant semantics - constant_or_zero - to look at test_brb test hangs JohnF - verifier loop support - on top of ast's call support tree - find basic blocks, build CFG - building dom tree - this should enable finding maximal bounds - goal to have something functional by end of the week Jesper - "XDP Infrastructure developments" at netconf http://vger.kernel.org/netconf2017part2.html http://people.netfilter.org/hawk/presentations/NetConf2017_Seoul/XDP_devel_update_NetConf2017_Seoul.pdf Daniel - Alexei+Daniel to manage new bpf tree - delegated from netdev - pull request ~1 per week - where do driver patches go? - to be discussed - bpf qdisc - replacement for clsact+bpf - compatible with clsbpf programs - removes a couple layers that aren't usually used - future: multiprogram support === Attendees === Brenden Blanco Alexander Duyck Daniel Borkmann Edwin Peer Jakub Kicinski Jesper Brouer Jiong Wang John Fastabend Kenny Paul Matteo Bertrone Mauricio Vasquez Quentin Monnet Yonghong Song Brendan Gregg BobB ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for attending the meeting today, here are my notes. === Status === Netdev 2.2 next week - Live streaming available - link to be sent out when available Brenden: - Increase cadence for updating package version - todo: update 0.4.1 Alexei: - Work proceeds on call between programs - bpf loader will start to look like a linker - still no loops - arbitrary argument definition JohnF: - work on sockmap - tcp hook - multiple cgroups - bounded for loops - rough sketch - detect nested loops = reject - finding linearly increasing/decreasing loops - finally, what is the bound of the loop - what about counters that are on stack instead of register? Yonghong: - fixing arm64 issues - netperf missing, other tests passing - uprobe performance improvements Martin: - added small bpf debugging tool to bcc Kenny: - TPM from LF - Hi! JohnB: - Also saying Hi from Netflix === Attendees === Jakub Kicinski Kenny Paul Marco Leogrande Panagiotis Moustafe Quentin Monnet Alexei Starovoitov Brenden Blanco John Bennett Alexander Duyck Yonghong Song Prem Nic Viljoen John Fastabend Teng Qin Kenny Paul Jesper Brouer ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for attending the call today. Below are my notes. === Status === Alexei: - 1 patch / day - burdensome on alexei/daniel - how to scale? - to be discussed at netconf - working on calls - can load relatively large files - rfc to come before netdev - memory pressure in the verifier - First TCP tracepoints accepted - 5 more coming soon - tcp set state has some corner case where it is missed Daniel: - fixes for verifier - llvm generates some compare/jumps that aren't covered - off by 1 in xdp data_end checks - lpm needs get next key helper for dumping to userspace - plan to support more drivers for xdp metadata Brendan: - LISA in a couple weeks - 2 bpf talks from Sasha, 1 from Brendan Jakub: - How to print 2nd phase bpf verifier output from offloaded program? - quirk: it runs at attach time, not load time - Maybe try a dry run somehow? - Intermixing of driver specific errors? - Same problem for maps - Include offload check at load time instead - New system call for "BPF_PROG_PREPARE", in between load and offload - jit a loaded bpf prog for a specific arch/offload === Attendees === Marco Leogrande Brenden Blanco Alexander Duyck Daniel Borkmann Jakub Kicinski Carlos Neira Quentin Monnet Alexei Starovoitov Jiong Brendan Gregg Mauricio Vasquez Nic Viljoen Gordon Marler ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi all, Thanks for attending the call today. Here are my notes. Also, please be advised that we will be moving to a 30 min time slot going forward. === Status === Yonghong: - Added source debugging printer for bcc - Requires 6.0 (possible but needs volunteers to backport to 5.0) - Permit multiple programs to attach to single tracepoint/probe - Uses ast's prog array for cgroups style attachment - Validating arm64 support - May help with arm32 support too Alexei: - Still working on calling routines (6 months and counting) - Big changes in verifier Jesper: - cpumap type available with xdp redirect Daniel: - Metadata->skb landed - Other drivers to come - Integrating xdp support into cilium John: - Fixes for sockmap - Adding support for ingress flags To-do list @ github.com/iovisor/bcc/issues/574 has been updated - stringmap and bounded loops are important - network protocol parsing - lots of looping requirements - http2, etc. Panagiotis (elasticcloud) - Interested in security (networking/sandboxing) - hitting limitations of seccomp-bpf and no pointer dereferences - is aware of landlock, to be looked into - should address needs Andy: - Close to adding bcm xdp_redirect suppport === Attendees === Brenden Blanco Marco Leogrande Quentin Monnet Jakub Kicinski Mauricio Vasquez Alexander Duyck Yonghong Song Alexei Starovoitov Andy Gospodarek Panagiotis Moustafellos Jesper Brouer Nic Viljoen Daniel Borkmann ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev meeting
Hi All, Thanks for attending the call yesterday, here are my notes: === News === Linux Plumbers was last week, and included an increasing focus on BPF for tracing and other use cases over previous years. The tracing/BPF track on Friday had a very packed agenda, only allowing 10-15 minutes per talk. For full notes on the discussions, please check out the etherpad at: https://etherpad.openstack.org/p/LPC2017_Tracing Other upcoming conferences: Kernel Recipes - Sept 27-29 https://kernel-recipes.org/en/2017/ Brendan - Performance Analysis with BPF Netdev 2.2 - Nov 8-10 https://netdevconf.org/2.2/ Last day to submit a talk was yesterday, registration is still open === Developer Status === Brenden - Work started on a LLVM Debug Info traverser for BCC - Should be able to generate CTF data Martin - Investigating CTF converter for use in kernel Jesper - new map type - cpu - like netdev map - redirecting to another cpu - example: ddos on first cpu - queue non-dropped packets to other cpu - other cpu allocs skb Martin - worked on tool to iterate over bpf programs and which maps they are using - Nic suggests a similar tool from Jakub to iterate programs - https://github.com/Netronome/bpf-tool Jiong - Working on 32 bit bpf backend in llvm - (makes netronome happy) - Please let the community know when something is ready and we can help with testing :) === Attendees === Panagiotis Moustafellos Nic Viljoen Mauricio Vasquez Martin Lau Marco Leogrande Jiong Jesper Brouer Jakub Kicinski Edwin Peer Brenden Blanco Brendan Gregg Alexander Duyck Abder ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for attending the call today. Here are the notes from the meeting. Netdev 1.2 reminder: paper submissions deadline is Sept 20 Linux Plumbers microconference (next week) If you have volunteered to lead discussion of a topic, Alexei would like to remind you that we don't want to be power pointed to death. If you have slides, they should be for fostering discussion, so please bring only a few :) We believe the session will be 2-5pm on Friday, but couldn't find confirmation on the website yet. === Status updates === Jakub: - created tool for bpf verifier disassembly - verifier -> elf - to be shared in near future William: - conn track RFC: feedback being addressed - wildcard map - looking at decision tree algorithm - some performance evaluation tbd - feedback: ok to make some problem space assumptions to simplify implementation complexity Yonghong: - looking at uprobe performance issues before plumbers - vdso, lttng - kernel tracepoints - only 1 id limitation - improvements? - bug with fd closing - pid inside container - helper to manage pid John: - Fixes for XDP - sockmap detach bug for net - redirect to ingress support for sockmap - parsing problems with complex protocols - fsm helper? Daniel: - Ran some statistics on bpf 4.0 -> 4.13 kernel commits - 123 patches (if the note-taker heard correctly) - 27 unique contributors - last 3 releases were busiest and growing - XDP to skb metadata transfer - tests and reviews Brendan: - bpf automation improvements @netflix - flamegraphs with oneclick Misc: - Landlock talk at security miniconf during plumbers Alexei: - focus is on visibility/debug - wants to resume function call work === Attendees === Alexei Starovoitov Andy Gospodarek Brendan Gregg Brenden Blanco Daniel Borkmann Flavio Crisciani Fulvio Russo Jakub Kicinski John Fastabend Mauricio Vasquez Suchakra Sharma William Tu Yonghong Song ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
Re: [iovisor-dev] minutes: IO Visor TSC/Dev Meeting
On Wed, Aug 23, 2017 at 4:40 PM, Brenden Blanco via iovisor-dev wrote: > Hi All, > > Thanks for joining in to the call today. We shared a round of updates, which > are > included in the notes below. > > Also please remember that Plumbers is right around the corner! If you have > signed up to lead discussion of a topic at the Tracing+BPF microconference, > Alexei would like to have a brief description of what will be discussed so > that > organizers can update the bylines on the schedule. Please send that soon! the proposals from tracing wiki need to be re-submitted to show up in official plumbers schedule. Please do so here: https://linuxplumbersconf.org/2017/ocw/events/LPC2017/tracks/640 Anything tracing or bpf related is a good topic to discuss. Thanks! ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for joining in to the call today. We shared a round of updates, which are included in the notes below. Also please remember that Plumbers is right around the corner! If you have signed up to lead discussion of a topic at the Tracing+BPF microconference, Alexei would like to have a brief description of what will be discussed so that organizers can update the bylines on the schedule. Please send that soon! John F: - currently working on improvements to sockmap - api changes merged - sockets can now be in multiple maps - bug: tracking down warning in socket cleanup Daniel: - currently working on misc net-next fixes - taking XDP measurements for early drop filters - next: XDP->CB metadata sharing Brenden: - working on some lib cleanups - starting investigation into including compact c types in maps for plumbers conf - note from Alexei: Martin is also looking at this, we should sync up - requests to share WIP patches for bpf+pktgen (done) Yonghong: - starting investigation into uprobe performance for plumbers - planning to also give llvm bpf compiler change overview William: - feature request: xdp_redirect with support for multi-dev output/multicast - response: has been discssed, and the feature makes sense - will take the form of a port group/multi-dev-map with kernel doing the duplication in the backend Joe: - working on rebasing bpf tuple space search table prototype - hitting gcc issues with container_of + rcu Attendees: Zhiting Zhu Yonghong Song William Tu John Fastabend Joe Stringer Flavio Crisciani Dongsu Park Daniel Borkmann Brenden Blanco Alexei Starovoitov ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for joining the meeting today. Here are the notes! === Updates === Yonghong: - allow >2 pids attach to same usdt probe - wip - next: looking at uprobe performance - looked into dyninst - researched systemtap approach - c prog compiled into .so - instrumentation of binary with jmp - limited helpers available - maps? - stacks? - filters in userspace - sample rate - malloc size - ongoing John - xdp redirect merged! - ongoing: some minor bugfixes - next: socket redirect Alastair: Welcome to the call! He is working on a new tool called bpftrace, which has a promising starting codebase. - new tool for tracing, standalone - check out github.com/ajor/bpftrace Daniel: - adjusted map value verifier fixes - iproute2 fixes Joe: - resuming work on ovs/bpf - looking at integrating cilium connection tracker with ovs-bpf !! New feature request: - add bpf id printing support to bcc === Attendees === Alastair Andy Gospodarek Brendan Gregg Brenden Blanco Daniel Borkmann Genevieve Bastien Jakub Kicinski Joe Stringer John Fastabend Marco Leogrande Mauricio Vasquez Yonghong Song ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for a great meeting today, lots of good discussions and updates! Here are the notes for those that missed it. Yonghong: LLVM fixes: - data/data_end comparison handling - read only section improved - proper handling of struct init - please pick up llvm 5.0 to get them (looking into backporting) upcoming: - loop unrolling (avoid instruction limits) - uprobe performance - in production = unusable - dyninst Martin: - bpf prog id - committed what next? - extra program info - function name from symbol table? - map ids used by program? - load time/age? - lru map - improve performance Brenden: - No updates, but requests from others for: - status on pktgen+bpf - static library on fedora linker issue Daniel: - mechanism to share xdp -> skb metadata - use case: extracting metadata from ip options - remove before gro - use in tc ingress, mark, etc. - discussion: scratch space for nic drivers? xdp programs for parsing similar use case from others on the call John: - redirect patch ongoing - how to handle m:n queueing/locking? - planning to submit as-is, unless the current approach has future uapi implications - good performance numbers Brendan: Plumbers/OSSNA: - giving talk on wed - added some topics to wiki, please add yours directly to the "Talk proposals" section Attendees: Yonghong Song Suchakra Sharma Nic Viljoen Mauricio Vasquez Martin KaFai Lau John Fastabend Jesper Brouer Jakub Kicinski Genevieve Bastien Edwinn Peer Daniel Borkmann Brenden Blanco Brendan Gregg Andy Gospodarek ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi all, Thanks for dialing in today, here are notes on what we discussed today. Upcoming conferences O'Reilly Velocity Brendan and Sasha both giving talks on BPF Open Source Summit North America Sept 11-14, Los Angeles 4 BPF/XDP talks accepted Linux Plumbers (shared/overlaps OSSNA) Sept 13-15, Los Angeles Submit your tracing microconference discussion topics! Also, be aware that registration fills early, and the early bird pricing window is almost closed (4 more days) Status Updates: Daniel: - ongoing kernel verifier fixes - register state/pruning in the verifier - next: additional verifier work - issue in bounds checking for adjusted map values - signed operations - implementing more compare instructions (to reduce register pressure) Brendan: - No dev work, but looking to collect more user stories around bpf - was able to trace fd allocation to identify a leak - excited about recent perf pmc+bpf improvements, but limited by ec2 support (available on some dedicated instances) Jesper: - diving into bpf map implementation (for port table) - getting receive bulking work for xdp (changed approach from previous) - based on j. fastabend's work Jakub: - Working on maps offload upstreaming (tricky, will come slowly) Carlos: - Would like to see bpf backported to 3.10/rhel7+arm32, wants to know if it is doable. Looking for help and feedback from the community. Attendees: Nic Viljoen Mauricio Vasquez Marco Leogrande Lorenzo David Jesper Brouer Edwin Peer Daniel Borkmann Carlos Neira Brenden Blanco Brendan Gregg Andy Gospodarek ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for joining the meeting today. Here are the notes! === Conferences === Reminder: CFP for Linux Plumbers Please submit a topic for the Tracing Microconference! http://wiki.linuxplumbersconf.org/2017:tracing === Technical Updates === Brenden: Doing some code cleanups for BCC - The goal will be to have cmake option flags so users of our api can pick the functionality desired (e.g. no clang for smaller footprint). - To start with, split libbpf into its own lib (just static for now, standalone dynamic to shipped when ready) - Expect source code/directory shuffling Todo: - improve native netlink support - add elf format support Why is this work important? Other libraries seem to be stalled, have incompatible licenses, or are entwined in their surrounding project. kernel: samples/bpf, libbpf iproute2 Usage of unicode/ascii strings in bcc tools has a PR pending, feedback from pythonistas would be appreciated. The bulk of the work remains in the form of removing encode/decode usages from the tools and testing on python3 enabled distros. Word on the call was that fedora is starting to package bcc with python3 as the default. Please file issues as found. Sidenote: usdt probes now available in python3.6 Alexei: - Stack reduction changes done - Moving along with function call changes - Q: How to understand capability in llvm+bcc? (will figure this out eventually) Jesper: - Feature bits for exposing hash data to xdp programs moving along, no rush - Working on xdp redirect help from Andy G? === Attendees: Alexander Duyck Alexei Starovoitov Andy Gospodarek Bilal Anwer Brenden Blanco Flavio Crisciani Jakub Kicinksi Jesper Brouer Martin Lau Mauricio Vasquez Nic Viljoen ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
Re: [iovisor-dev] minutes: IO Visor TSC/Dev Meeting
On Wed, May 17, 2017 at 2:14 PM, Brenden Blanco via iovisor-dev wrote: > Hi All, > > Thanks for attending the call today, here are the notes. thanks a lot for taking and publishing these notes! > CFP for Linux Plumbers > Sept 13-15, Los Angeles > Tracing microconference accepted > (Alexei, Brendan, Josef leading topics) > http://wiki.linuxplumbersconf.org/2017:tracing > Last year was successful, looking forward to more topics > (tbd: find previous etherpad) https://etherpad.openstack.org/p/LPC2016_Tracing a bit terse, but if you were there last year, it's a good memory refresh :) ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for attending the call today, here are the notes. == CFP for Linux Plumbers Sept 13-15, Los Angeles Tracing microconference accepted (Alexei, Brendan, Josef leading topics) http://wiki.linuxplumbersconf.org/2017:tracing Last year was successful, looking forward to more topics (tbd: find previous etherpad) == BCC has grown, albeit with some maturity/sore spots We have a well established core group, but probably need to expand our feedback to incorporate those that may not be speaking up. Think users of the tools rather than developers. How do we get broader user-group feedback besides mailing list/github issues? Brendan: #1 feedback he has heard - simplify syntax, still compared to dtrace awareness via conferences, etc. Alexei: internally, writing programs is hard, struggling with verifier - working on llvm improvements many kernel hackers @fb have hacked up bcc tools for their own use case Brenden: reach larger user group => collectd integration? command line users are the smaller subset of our potential audience intel snapd collectd others? Going forward: It's a big library, needs shrink/split both in terms of memory usage and disk size split out clang/llvm split out loader AI: get some proposals into bcc/issues, either in documentation form to start with, or code split header files split libraries start thinking about external tool integration == Alexei: has started thinking about program chaining (generically) mistake: tail call solved a need, but helper/map model makes it inflexible prog chaining should be a single native instruction (not done until now because "too hard to verify") now should be possible to introduce and verify cleanly direct, indirect, and tail call support (stack preserving) 3 new bpf instructions introduce global variables verifier changes started llvm support (looks like normal c) configurable stack upper bound of function loader would need to parse elf file main() function concept, still callable via tail call So far, the above is perhaps 10% compelete, so stay tuned. == Attendees: Alexei Starovoitov Bilal Anwer Jesper Brouer Martin Lau Mauricio Vasquez Brenden Blanco Andy Gospodarek Nic Viljoen Daniel borkmann Suchackra Sharma Brendan Gregg John Fastabend Teng Qin Jakub Kicinski Marco Leogrande ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for joining the meeting this week. Here are my notes from the meeting. See you all in two weeks! === Conferences === Notes from Netdev 2.1 Lots of focus on BPF, XDP infra and use cases Dave Miller: XDP Keynote working on generic netdev xdp support xdp-newbies mailing list Jesper + Andy: - tutorial on getting up and running on xdp - typical bpf pitfalls Jesper: - some hallway discussions to drop one-page-per-packet requirement - todo: write-up intro xdp doc - common thread: every nic vendor wants xdp support - work on changing to RW pages in all drivers - speeds up normal stack - uncomplicates some extra allocations - avoids some copying Daniel: - cloudflare use case to transfer xdp data to skb - i40e rework is in progress - discussions on hash during jit for hardening -> NACKed --- Dockercon - Two presentations that are BPF/XDP related - Cilium - well received talk - http://blog.scottlowe.org/2017/04/18/black-belt-cilium/ - Brendan Gregg - Container Performance Analysis - https://www.slideshare.net/brendangregg/container-performance-analysis - Also announced: linuxkit - small linux distro, hardened (e.g. landlock, bcc scripts, okernel/moby, wireguard) === Technical Updates === Brenden: Investigating bpf hook for pktgen - Use case is to customize filling of packet contents - pktgen generates empty frame with specified headroom - bpf program uses xdp helpers to adjust packet size and write contents William: progress on hitting verifier limits with complex program clang unrolling issues (worked around) array initializer issues (check readelf to see if extra sections present) LLVM needs some fixes to do these properly Daniel: fixes for tailcalls socket filter, xdp adjust head Suchakra: Tracing with ctf (branch), babeltrace bindings for python Adding in native ctf support to avoid dependency tool vcpu usage from VM via kvm tracepoints Martin: map-in-map work upstreamed for LB use case working on bpf program/map id wip patch coming soon Attendees: William Tu Suchakra Sharma Nic Viljoen Mauricio Vasquez Martin Lau Joe Stringer Jesper Brouer Jakub Kicinski dsrl_ Daniel Borkmann Chris Brenden Blanco Anirban Andy Gospodarek ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi All, Thanks for joining the meeting this week. Here are some status updates from various folks. William: Discussed issue he is having with the verifier/llvm complex program becomes stack limited as a workaround, tried using volatile modifier for some stack variables but this causes the verifier to hit the 64k simulated instruction complexity limit reason: no heuristics to prune states involving on-stack variables suggested workaround: use per-cpu array to hold state Brenden: Saw issue with xdp in virtio warn on once in drivers/pci/msi.c:1261 pci_irq_vector+0xd4/0xe0 will send description to netdev list ctf (tracing and types) topics: Suchakra (tracing) - started with babeltrace python bindings - discussed optional compile-time dependency on libbabeltrace - we agreed for now to work on a separate bcc branch for integration testing Alexei: libraries exist as part of pahole (maintained by Arnaldo): payhole is tool for analyzing holes in c datastructures supports dwarf+ctf parser and emitter may need to write emitter from scratch due to licensing, but not too hard bitfield representation may need some work api: extra section at map create time ability to dump as well via syscall new name to avoid ambiguity with tracing-ctf? Andy: arm64 compiler issues inline assembly parser in llvm must-have for tracing to get accurate header definitions open PR already exists for hacking it in BCC to ifdef out inline assembly (ugly) suggestion to respin patches to arm list to cleanup defines Sargun: new landlock patches a month ago, progress? some discussion, seems stalled (unprivileged requirement) alternative: seccomp syscall filtering at global level (with eBPF) avoids criu problem Attendees: William Tu Teng Suchakra Sharma Sargun Dhillon Michael Schubert Mauricio Vasquez Matteo Bertrone Martin Lau Marco Leogrande Lorenzo David Joe Stringer Jan-Erik Jakub Kicinski Gary Berger Flavio Crisciani Ferris Daniel Borkmann Brenden Blanco Brendan Gregg Andy Gospodarek Alexei Starvoitov ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
Hi all, Thanks to those who attended, here are my minutes from the meeting. = Conferences past and future = IO Visor Summit We had this last week, the conference room was packed and we had a few remote attendees as well. We are working to upload the videos, collect the slides (please send them to Debra if you haven't already!), and maybe write a blog post. If you weren't able to attend, look out for an email shortly. SCaLEx 15x Linux 4.x Tracing: Performance Analysis with bcc/BPF https://www.socallinuxexpo.org/scale/15x/presentations/linux-4x-tracing-performance-analysis-bccbpf Brendan gave this featured talk, with an estimated 200-300 attendees in the room. Many people were surprised and impressed with the capabilities, which means that there are still too many people who don't know about BPF yet! Get out there and give talks if you can. O'Reilly Velocity - June 19-22 - Santa Clara, CA Accepted talk by Brendan, and tutorial by Sasha Netdev 2.1 - April 6-8 - Montreal, CA CFP is over, several BPF/XDP topics on the agenda Jesper: Tutorial on how to program BPF Dockercon - April 17-20 - Austin, TX Thomas: Cilium - Network and Application Security with BPF and XDP Brendan: Container Performance Analysis (not BPF specific) Other (I didn't catch the name) - Denmark Jesper: how to use bpf for ddos mitigation = Development = Brenden: Working on changing the way bpf maps are stored in the BPFModule and related classes, with the hope of having different backend implementations based on feature requirements (persistence, namespaces). The first user of the change should be the json type generator, followed by the MapTypesVisitor proposal (bcc #960). CTF Discussion kicked off regarding encoding of types. The use of CTF (compact C type format) has been mentioned before, and still seems to be the way to go. Alexei will kick off a discussion thread with some kernel folks who supposedly have some baseline code work off of. Suchakra: Working on something sip (voice protocol) related, details to come soon Some ctf (t=trace) work ongoing, depending on Babeltrace 2.0 = Attendees = Brenden Blanco Joe Stringer Marco Leogrande Suchakra Brendan Gregg Alexei Starovoitov Matteo Bertrone Mauricio Vasquez William Tu ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
[iovisor-dev] minutes: IO Visor TSC/Dev Meeting
IO Visor summit coming, please prepare abstracts if you have something to discuss === Conferences === netdev conf coming up, talks needed Jesper - performance workshop DDoS topic idea: reach out to nic vendors who support xdp - future xdp direction idea: dpdk work idea: specifically, xdp ring to userspace john's proposal for sharing pages FOSDEM 2017 4 different BPF related talks: https://fosdem.org/2017/schedule/event/cilium/ https://fosdem.org/2017/schedule/event/ebpf_xdp/ https://fosdem.org/2017/schedule/event/stateful_ebpf/ https://fosdem.org/2017/schedule/event/go_bpf/ linux.conf.au BPF: Tracing and More https://linux.conf.au/schedule/presentation/96/ BSidesSF Linux Monitoring at Scale with eBPF http://sched.co/9IKj === Dev Updates === bcc - buildbot/jenkins moved over to GCE slower but cheaper/on demand Daniel: tracepoints for bpf accepted xdp support (new drivers, new features) verifier fixes kallsym work wrapping up lpm trie features and fixes Jesper: documentation work goal to enable customers to create own DDoS filters instead of RH coding on its own cgroup work: netns id helper William: hitting max stack size (check printk strings) unlikely to increase in kernel === Attendees === Daniel Borkmann Eduard Serra Brenden Blanco Marco Leogrande Joe Stringer Davide Celotto Mauricio Vasquez Michael Schubert William Tu Jesper Brouer Brendan Gregg Sargun Dhillon ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
