Hi Willy, I've managed to implement the easy part (attached patches). However this is not enough to get it to work properly. I'm stuck in a debug of following stack trace (BUG_ON task_queue() function):
FATAL: bug condition "task->thread_mask != tid_bit" matched at include/haproxy/task.h:320 call trace(12): | 0x55f921169be3 [c7 04 25 01 00 00 00 00]: main+0x112f53 | 0x55f92116c4ee [48 8b 43 08 48 8b 78 10]: si_applet_wake_cb+0x7e/0xf3 | 0x55f9211cc276 [48 8b 43 08 48 8b 40 10]: task_run_applet+0x196/0x5bd | 0x55f92118e6ca [48 89 c5 eb 0f 90 4c 89]: run_tasks_from_lists+0x3aa/0x871 | 0x55f92118ef73 [29 44 24 14 8b 7c 24 14]: process_runnable_tasks+0x3d3/0x892 | 0x55f92115eb44 [83 3d f9 ef 1d 00 01 0f]: run_poll_loop+0x124/0x3d4 | 0x55f92115efc9 [48 8b 1d d0 94 12 00 4c]: main+0x108339 | 0x55f9210589ff [31 c0 e8 ba a6 16 00 31]: main+0x1d6f/0x296a I'm not sure why it happens. I suspect that it happens because of session_new() in peer_session_create() that creates another task on the current thread. If a session task tries to wake up a peer applet task then it would explain the stack trace, however I've failed to fully understand the "flow" between applet and session (and how/when task_queue is called in this flow). I would appreciate any help or tips on how to proceed. Kind regards, pt., 1 kwi 2022 o 14:34 Willy Tarreau <[email protected]> napisał(a): > On Fri, Apr 01, 2022 at 11:23:34AM +0200, Maciej Zdeb wrote: > > > I remember mentioning something about trying to change this in the > future. > > > For outgoing connections I think we could try to figure where peers > > > connections are and always use one least represented thread. > > > > > If I understand the code correctly, the peer_session_create() function > > creates appctx by appctx_new() which allocates all outgoing connections > on > > the current thread "appctx->t = task_new_here();"? > > So changing peer_session_create() into: > > > > static struct appctx *peer_session_create(struct peers *peers, struct > peer > > *peer, int thread) { > > ... > > appctx = appctx_new_on(&peer_applet, thread); > > if (!appctx) > > goto out_close; > > ... > > } > > > > where appctx_new_on() is a new function that creates an applet on a > > specified thread would fix my issue? I'm worrying it's not that easy and > > some locking here and there might be needed. ;-) > > Indeed, I think it would work without problems! In this case you could > even add an array of MAXTHREADS to the peers struct to count the number > of applets per thread, then peer_session_create() could just scan the > threads to find the least loaded one and use this one. This would easily > do the job and sounds simple and efficient enough to get into 2.6. > > Please, if you go that way, think about splitting your work in at least > 3 patches: > - 1 for appctx_new_on() > - 1 to maintain the applet count per peers struct > - 1 to use that counter to start the applet on the best thread > > You may find that you need to implement more stuff, but you get the idea. > > thanks, > Willy >
0001-MINOR-applet-Add-function-to-create-applet-on-select.patch
Description: Binary data
0002-MINOR-peers-Track-number-of-applets-run-by-thread.patch
Description: Binary data
0003-MINOR-peers-Balance-applets-across-threads.patch
Description: Binary data
