回复：Attach to shared memory after fork()
> Windows has CreateProcess, which isn't available elsewhere. Yes, we still need fork() on *nix. So the solution is to reduce the overhead of fork(). Attach to shared memory after fork() might be a "Better shared memory management". > This is one of the reasons for using a connection pooler like pgbouncer, > which can vastly reduce the number of new process creations Postgres has to do. Yes, it’s another way I forgot to mention. But I think there should be a cleaner way without other component. > This proposal seems moderately insane. In the first place, it > introduces failure modes we could do without, and in the second place, > how is it not strictly *more* expensive than what happens now? You > still have to end up with all those TLB entries mapped in the child. Yes, the idea is radical. But it’s practical. 1. I don’t quite catch that. Can you explain it? 2. Yes, the overall cost is still the same, but the cost can spread into multi processes thus CPUs, not 100% on Postmaster. > (If your kernel is unable to pass down shared-memory TLBs effectively, > ISTM that's a kernel shortcoming not a Postgres architectural problem.) Indeed, it’s a kernel/CPUarch shortcoming. But it is also a Postgres architectural problem. MySQL and Oracle have no such problem. IMHO Postgres should manage itself well(eg. IO/buffer pool/connection/...) and not rely so much on OS kernel... The fork() used to be a genius hack, but now it’s a burden and it will get worse and worse. All I want to do is remove fork() or reduce the overhead. Maybe *nux will have CreateProcess someday(and I think it will), and we should wait for it?
Re: Attach to shared memory after fork()
"=?UTF-8?B?6YKx5a6H6IiqKOeDm+i/nCk=?=" writes: > Fork is an expensive operation[1]. Yeah, it's not hugely cheap. > So I propose to remove shared buffers from postmaster and shmat them > after fork. This proposal seems moderately insane. In the first place, it introduces failure modes we could do without, and in the second place, how is it not strictly *more* expensive than what happens now? You still have to end up with all those TLB entries mapped in the child. (If your kernel is unable to pass down shared-memory TLBs effectively, ISTM that's a kernel shortcoming not a Postgres architectural problem.) regards, tom lane
Re: Attach to shared memory after fork()
On 4/26/21 11:56 PM, 邱宇航(烛远) wrote: > Fork is an expensive operation[1]. The major cost is the mm(VMA > PTE...) copy. > > ARM is especially weak on fork, which will invalid TLB entries one by > one, and this is an expensive operation[2]. We could easily got 100% > CPU on ARM machine. We also meet fork problem in x86, but not as > serious as arm. > > We can avoid this by enable hugepage(and 2MB doesn’t help us under > arm, we got a huge shared buffer), but we still think it is a problem. > > So I propose to remove shared buffers from postmaster and shmat them > after fork. Not all of them, we still keep necessary shared memories > in postmaster. Or maybe we just need to give up fork like under Windows? > Windows has CreateProcess, which isn't available elsewhere. If you build with EXEC_BACKEND on *nix it will fork() followed by exec(), the classic Unix pattern. You can benchmark that but I doubt you will like the results. This is one of the reasons for using a connection pooler like pgbouncer, which can vastly reduce the number of new process creations Postgres has to do. Better shared memory management might be more promising. cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com
Attach to shared memory after fork()
Fork is an expensive operation[1]. The major cost is the mm(VMA PTE...) copy. ARM is especially weak on fork, which will invalid TLB entries one by one, and this is an expensive operation[2]. We could easily got 100% CPU on ARM machine. We also meet fork problem in x86, but not as serious as arm. We can avoid this by enable hugepage(and 2MB doesn’t help us under arm, we got a huge shared buffer), but we still think it is a problem. So I propose to remove shared buffers from postmaster and shmat them after fork. Not all of them, we still keep necessary shared memories in postmaster. Or maybe we just need to give up fork like under Windows? Any good idea about it? [1]. https://www.microsoft.com/en-us/research/publication/a-fork-in-the-road/ [2]. https://developer.arm.com/documentation/ddi0487/latest/ D5.10 TLB maintenance requirements and the TLB maintenance instructions: Break-before-make sequence on changing from an old translation table entry to a new translation table entryrequires the following steps: 1. Replace the old translation table entry with an invalid entry, and execute a DSB instruction. 2. Invalidate the translation table entry with a broadcast TLB invalidation instruction, and execute a DSBinstruction to ensure the completion of that invalidation. 3. Write the new translation table entry, and execute a DSB instruction to ensure that the new entry is visible. Regards. Yuhang Qiu.
