I got the idea from https://people.mpi-sws.org/~druschel/publications/soft-timers-tocs.pdf The gist is that the TCP pacing stuff needs to run frequently, and rather than run it out of a clock interrupt, its more efficient to run it out of a system call context at just the point where we return to userspace and the cache is trashed anyway. The current implementation is fine for our workload, but probably not idea for a generic system. Especially one where something is banging on system calls.
Ast's could be the right tool for this, but I'm super unfamiliar with them, and I can't find any docs on them. Would ast_register(0, ASTR_UNCOND, 0, func) be roughly equivalent to what's happening here? Drew On Mon, Mar 18, 2024, at 2:33 PM, Konstantin Belousov wrote: > On Mon, Mar 18, 2024 at 07:26:10AM -0500, Mike Karels wrote: > > On 18 Mar 2024, at 7:04, tue...@freebsd.org wrote: > > > > >> On 18. Mar 2024, at 12:42, Nuno Teixeira <edua...@freebsd.org> wrote: > > >> > > >> Hello all! > > >> > > >> It works just fine! > > >> System performance is OK. > > >> Using patch on main-n268841-b0aaf8beb126(-dirty). > > >> > > >> --- > > >> net.inet.tcp.functions_available: > > >> Stack D Alias PCB > > >> count > > >> freebsd freebsd 0 > > >> rack * rack 38 > > >> --- > > >> > > >> It would be so nice that we can have a sysctl tunnable for this patch > > >> so we could do more tests without recompiling kernel. > > > Thanks for testing! > > > > > > @gallatin: can you come up with a patch that is acceptable for Netflix > > > and allows to mitigate the performance regression. > > > > Ideally, tcphpts could enable this automatically when it starts to be > > used (enough?), but a sysctl could select auto/on/off. > There is already a well-known mechanism to request execution of the > specific function on return to userspace, namely AST. The difference > with the current hack is that the execution is requested for one callback > in the context of the specific thread. > > Still, it might be worth a try to use it; what is the reason to hit a thread > that does not do networking, with TCP processing? > > > > > Mike > > > > > Best regards > > > Michael > > >> > > >> Thanks all! > > >> Really happy here :) > > >> > > >> Cheers, > > >> > > >> Nuno Teixeira <edua...@freebsd.org> escreveu (domingo, 17/03/2024 à(s) > > >> 20:26): > > >>> > > >>> Hello, > > >>> > > >>>> I don't have the full context, but it seems like the complaint is a > > >>>> performance regression in bonnie++ and perhaps other things when > > >>>> tcp_hpts is loaded, even when it is not used. Is that correct? > > >>>> > > >>>> If so, I suspect its because we drive the tcp_hpts_softclock() routine > > >>>> from userret(), in order to avoid tons of timer interrupts and context > > >>>> switches. To test this theory, you could apply a patch like: > > >>> > > >>> It's affecting overall system performance, bonnie was just a way to > > >>> get some numbers to compare. > > >>> > > >>> Tomorrow I will test patch. > > >>> > > >>> Thanks! > > >>> > > >>> -- > > >>> Nuno Teixeira > > >>> FreeBSD Committer (ports) > > >> > > >> > > >> > > >> -- > > >> Nuno Teixeira > > >> FreeBSD Committer (ports) > > >
