Re: gpg-agent "forgetting" keys when getting many parallel requests
On Mon Mar 18, 2024 at 09:50, Werner Koch wrote: > On Sun, 17 Mar 2024 13:09, Bence Ferdinandy said: > > > running out of memory. Based on a discussion I found > > (https://dev.gnupg.org/T4255), I set `auto-expand-secmem 100M` in > > Right. The man page says: > > --auto-expand-secmem n > >Allow Libgcrypt to expand its secure memory area as required. >The optional value n is a non-negative integer with a suggested >size in bytes of each additionally allocated secure memory area. >The value is rounded up to the next 32 KiB; usual C style >prefixes are allowed. For an heavy loaded gpg-agent with many >concurrent connection this option avoids sign or decrypt errors >due to out of secure memory error returns. > > You should not append the 'M' - it is simply ignored. That is a bug in > the option parser but we can't fix that because it would break too many > configs which falsely assume that a letter can be used for some kind of > unit. > > The value is actually irrelevant becuase any value will enable the > auto-expand behaviour. Larger chunks can make maneory allocation a biut > faster because every free() call needs to check the linked list of > secure memory pools. I am not sure whetehr this is measurable, though. Thanks for the clarification! Best, Bence ___ Gnupg-users mailing list Gnupg-users@gnupg.org https://lists.gnupg.org/mailman/listinfo/gnupg-users
Re: gpg-agent "forgetting" keys when getting many parallel requests
On Sun, 17 Mar 2024 13:09, Bence Ferdinandy said: > running out of memory. Based on a discussion I found > (https://dev.gnupg.org/T4255), I set `auto-expand-secmem 100M` in Right. The man page says: --auto-expand-secmem n Allow Libgcrypt to expand its secure memory area as required. The optional value n is a non-negative integer with a suggested size in bytes of each additionally allocated secure memory area. The value is rounded up to the next 32 KiB; usual C style prefixes are allowed. For an heavy loaded gpg-agent with many concurrent connection this option avoids sign or decrypt errors due to out of secure memory error returns. You should not append the 'M' - it is simply ignored. That is a bug in the option parser but we can't fix that because it would break too many configs which falsely assume that a letter can be used for some kind of unit. The value is actually irrelevant becuase any value will enable the auto-expand behaviour. Larger chunks can make maneory allocation a biut faster because every free() call needs to check the linked list of secure memory pools. I am not sure whetehr this is measurable, though. Salam-Shalom, Werner -- The pioneers of a warless world are the youth that refuse military service. - A. Einstein openpgp-digital-signature.asc Description: PGP signature ___ Gnupg-users mailing list Gnupg-users@gnupg.org https://lists.gnupg.org/mailman/listinfo/gnupg-users
Re: gpg-agent "forgetting" keys when getting many parallel requests
On Fri Mar 15, 2024 at 20:16, Bence Ferdinandy wrote: > Hi, > > I've been poking at this issue now for some time, and haven't been able to > find > any info regarding this in the man pages or online. > > I'm running Regolith (basically Ubuntu with i3), and normally my gpg key is > unlocked by me logging into the graphical session, i.e. using the key to > decrypt something does not ask for a key. What I noticed though, is that if > I want to decrypt multiple things in parallel, first the decryption process > seems to slow down faster than linear, and then some (not all) decrypt tasks > start prompting me for a password to the key. I've done a couple of > experiments, and my feeling is that even though the decrypt task are run in > parallel, something is done serially, but I'm not sure what and where exactly > and that there is a timeout on waiting for this. My best bet, is that the > password for the key needs to be fetched from the gnome keyring (? if it's > called that) and that gpg-agent times out waiting for this and just requests > it > from the user. > > I made a short script in python (attached) demonstrating this. On my machine, > setting WORKERNUM=7 is enough to trigger the issue. > > Could somebody point me to a resources explaining what is happening here? And > more importantly, is there some setting I can change to avoid the occasional > password prompts? After looking at logs even more, I found that actually gpg-agent reports running out of memory. Based on a discussion I found (https://dev.gnupg.org/T4255), I set `auto-expand-secmem 100M` in gpg-agent.conf which seems to have resolved the issue even with 100 parallel threads. I have no idea if this 100M is way too much or not, but I haven't found any mentions of a downside to such a setting and it seems to work as expected, so I'm happy for now. Best, Bence ___ Gnupg-users mailing list Gnupg-users@gnupg.org https://lists.gnupg.org/mailman/listinfo/gnupg-users
gpg-agent "forgetting" keys when getting many parallel requests
Hi, I've been poking at this issue now for some time, and haven't been able to find any info regarding this in the man pages or online. I'm running Regolith (basically Ubuntu with i3), and normally my gpg key is unlocked by me logging into the graphical session, i.e. using the key to decrypt something does not ask for a key. What I noticed though, is that if I want to decrypt multiple things in parallel, first the decryption process seems to slow down faster than linear, and then some (not all) decrypt tasks start prompting me for a password to the key. I've done a couple of experiments, and my feeling is that even though the decrypt task are run in parallel, something is done serially, but I'm not sure what and where exactly and that there is a timeout on waiting for this. My best bet, is that the password for the key needs to be fetched from the gnome keyring (? if it's called that) and that gpg-agent times out waiting for this and just requests it from the user. I made a short script in python (attached) demonstrating this. On my machine, setting WORKERNUM=7 is enough to trigger the issue. Could somebody point me to a resources explaining what is happening here? And more importantly, is there some setting I can change to avoid the occasional password prompts? Thanks, Bence import subprocess import multiprocessing as mp import time the_queue = mp.Queue() WORKERNUM = 7 def worker_main(queue): while True: msg = queue.get(True) print(time.time(), msg) out = subprocess.run(["gpg", "--decrypt", "test.gpg"], capture_output=True) print(msg, time.time(), out.stdout) the_pool = mp.Pool(WORKERNUM, worker_main, (the_queue,)) counter = 0 while True: counter += 1 the_queue.put(counter) print(the_queue.qsize()) while the_queue.qsize() > 10: time.sleep(0.1) ___ Gnupg-users mailing list Gnupg-users@gnupg.org https://lists.gnupg.org/mailman/listinfo/gnupg-users