Re: Multithreading with GPGME Python Bindings
(Please forgive the HTML email, sending from my phone)Given Python is effectively single-threaded through the global interpreter lock, this may turn out to be a total non-issue. Although I don't have an immediate answer for you I'd suggest starting by learning how Python's multi-threading support is more smoke and mirrors than reality. That may lead you to an answer to your question.On Jan 16, 2023 1:55 AM, Jan Girlich wrote:Hi, I want to use the GPGME Python bindings in a concurrent way. In the documentation of the Python bindings at http://files.au.adversary.org/crypto/gpgme-python-howto.html I find no mention of multithreading. In the GPGME documentation at https://www.gnupg.org/documentation/manuals/gpgme/Multi_002dThreading.html#Multi_002dThreading I find the note, that I need to call "gpgme_check_version" before doing any multithreaded operations. Then again I find only this section about version checking in the Python binding's documentation: http://files.au.adversary.org/crypto/gpgme-python-howto.html#gpgme-version-check In this section the version check is done via a subprocess call to "gpgme-config --version". Is the call to "gpgme_check_version" maybe done implicitly by the Python bindings? Do I have to call "gpgme-config --version" before to safely do multithreaded operations with the Python bindings? Thank you for any insights, Jan ___ Gnupg-users mailing list Gnupg-users@gnupg.org https://lists.gnupg.org/mailman/listinfo/gnupg-users ___ Gnupg-users mailing list Gnupg-users@gnupg.org https://lists.gnupg.org/mailman/listinfo/gnupg-users
Re: Multithreading with GPGME Python Bindings
Hi Robert, On Mon, 2023-01-16 at 11:53 -0500, Robert J. Hansen wrote: > I don't have an immediate answer for you I'd suggest starting by > learning how Python's multi-threading support is more smoke and > mirrors than reality. That may lead you to an answer to your > question. you seem to be quite new to the topic of Python and multi-threading, so let me give you the relevant basics relating to my question. Yes, the GIL is making cPython thread-safe and thus blocking threads. But only when processing Python bytecode. When an external operation runs, like gnupg encrypting something or I/O operations, the GIL is released and another thread can run. So multi-threading computationally intensive gnupg operations can have performance improvements. Also, the GIL is no issue with multi-processing, which is the constructive answer I prefer to "GIL is a problem", instead of "smoke and mirrors". I can recommend https://superfastpython.com/multiprocessing-pool-gil/ on this topic. Thank you, Jan > > On Jan 16, 2023 1:55 AM, Jan Girlich wrote: > > Hi, > > > > I want to use the GPGME Python bindings in a concurrent way. > > > > In the documentation of the Python bindings at > > http://files.au.adversary.org/crypto/gpgme-python-howto.html I find > > no > > mention of multithreading. > > > > In the GPGME documentation at > > https://www.gnupg.org/documentation/manuals/gpgme/Multi_002dThreading.html#Multi_002dThreading > > I find the note, that I need to call "gpgme_check_version" before > > doing > > any multithreaded operations. > > > > Then again I find only this section about version checking in the > > Python binding's documentation: > > http://files.au.adversary.org/crypto/gpgme-python-howto.html#gpgme-version-check > > In this section the version check is done via a subprocess call to > > "gpgme-config --version". > > > > Is the call to "gpgme_check_version" maybe done implicitly by the > > Python bindings? > > > > Do I have to call "gpgme-config --version" before to safely do > > multithreaded operations with the Python bindings? > > > > Thank you for any insights, > > Jan > > > > ___ > > Gnupg-users mailing list > > Gnupg-users@gnupg.org > > https://lists.gnupg.org/mailman/listinfo/gnupg-users > > > > (Please forgive the HTML email, sending from my phone) > > Given Python is effectively single-threaded through the global > interpreter lock, this may turn out to be a total non-issue. Although > I don't have an immediate answer for you I'd suggest starting by > learning how Python's multi-threading support is more smoke and > mirrors than reality. That may lead you to an answer to your > question. > > On Jan 16, 2023 1:55 AM, Jan Girlich wrote: > > Hi, > > > > I want to use the GPGME Python bindings in a concurrent way. > > > > In the documentation of the Python bindings at > > http://files.au.adversary.org/crypto/gpgme-python-howto.html I find > > no > > mention of multithreading. > > > > In the GPGME documentation at > > https://www.gnupg.org/documentation/manuals/gpgme/Multi_002dThreading.html#Multi_002dThreading > > I find the note, that I need to call "gpgme_check_version" before > > doing > > any multithreaded operations. > > > > Then again I find only this section about version checking in the > > Python binding's documentation: > > http://files.au.adversary.org/crypto/gpgme-python-howto.html#gpgme-version-check > > In this section the version check is done via a subprocess call to > > "gpgme-config --version". > > > > Is the call to "gpgme_check_version" maybe done implicitly by the > > Python bindings? > > > > Do I have to call "gpgme-config --version" before to safely do > > multithreaded operations with the Python bindings? > > > > Thank you for any insights, > > Jan > > > > ___ > > Gnupg-users mailing list > > Gnupg-users@gnupg.org > > https://lists.gnupg.org/mailman/listinfo/gnupg-users > > > ___ Gnupg-users mailing list Gnupg-users@gnupg.org https://lists.gnupg.org/mailman/listinfo/gnupg-users
How to make remote gpg talk to specific Unix socket (for forwarded local gpg-agent)
Hi, I was wondering if you could give me a pointer. I'm ssh'ing to a machine where I'm trying to run gpg, which I hope to talk to the gpg-agent that's running on my local laptop, forwarded through ssh. I'm following the instructions at https://wiki.gnupg.org/AgentForwarding but I don't know how to make gpg use a specific Unix socket to communicate with my forwarded gpg-agent. The problem is that there's already a gpg-agent running at the remote (I think started by systemd or the X server), which I don't want to disturb. For my ssh connection, I try to create a new Unix domain socket by using ssh -R /run/user/1000/gnupg/S.gpg-agent.remote:/Users/troy/.gnupg/S.gpg-agent.extra -o StreamLocalBindUnlink=yes -o ExitOnForwardFailure=yes But then how do I make gpg use/run/user/1000/gnupg/S.gpg-agent.remote . It seems that gpg uses gpgconf to figure out that path. And there's no flag or envvar that I can use to override that (anymore). The only thing I can think of is to specify a new --homedir so that the socket is created elsewhere, and then I'd have to change the port forwarding to reflect the dynamically-generated path which will look something like /run/user/1000/gnupg/d.xhmoxiusfxtwuy8s69hkyxtc . So either I have to do two separate ssh calls or I have to use an expect script to automate the addition of port forwarding on an existing session. Plus, now the homedir is different and all my files are in the wrong place. I hope I'm missing something obvious. Thanks for any ideas. I'm using: - Ubuntu 22.04.1 LTS - gpg (GnuPG) 2.2.27 - OpenSSH_8.9p1 Ubuntu-3ubuntu0.1, OpenSSL 3.0.2 15 Mar 2022 Troy ___ Gnupg-users mailing list Gnupg-users@gnupg.org https://lists.gnupg.org/mailman/listinfo/gnupg-users
Re: En-/Decryption speed for large files (GnuPG and Gpg4win)
Thanks for your replies! On 15.01.23 16:14, Ming Kuang wrote: gpg --decrypt test_file.gpg without output file will print all the decrypted contents on the screen, which may be the reason why it takes so long. For some reason in that test gpg didn't output anything or at least the PowerShell didn't show anything. On 15.01.23 17:03, Werner Koch wrote: > BTW, Do not use gpg4win 4.0.3 - it has a known vulnerability. Use > gpg4win 4.1.0. This will also change the numbers because we improved > some things in gpg. Don't worry, the system is mostly offline ;-) When I will give it access to the internet again I will update Gpg4win. Anyway, great to hear that the current version is faster than 4.0.3. On 16.01.23 02:01, Ángel wrote: > For any test like this where you are not going to process the output > (e.g. to compare it) I would recommend writing into the null device > (/dev/null in *nix, nul in Windows). I wanted to create an output file because I wanted to see how GnuPG would behave in a real scenario :) > Also, when measuring encryption make sure it is not trying to use > compression (based on the preferences of your test key). The time spent > by the compressor on your uncompressible files would be just an > unneeded source of variation. Thanks for the hint! I will try it with disabled compression. With regards, Christoph -- Christoph Klassen | https://intevation.de Intevation GmbH, Osnabrück, DE; Amtsgericht Osnabrück, HRB 18998 Geschäftsführer: Frank Koormann, Bernhard Reiter OpenPGP_signature Description: OpenPGP digital signature ___ Gnupg-users mailing list Gnupg-users@gnupg.org https://lists.gnupg.org/mailman/listinfo/gnupg-users
RE: En-/Decryption speed for large files (GnuPG and Gpg4win)
On Monday, January 16, 2023 9:02 AM, ángel wrote: > On 2023-01-15 at 23:14 +0800, Ming Kuang via Gnupg-users wrote: > > On Sunday, January 15, 2023 5:52 PM, Christoph Klassen wrote: > > > When I was testing the decryption I also tried "gpg --decrypt > > > test_file.gpg" (without output file) with the 10 GB file and it took 8 > > > minutes and 47 seconds. I was wondering why it took longer when GnuPG > > > didn't need to create an output file. > > > > As far as I know, outputting text to the screen (like printf) is a very time > > consuming operation, it will block you until all printing is complete. > > > > gpg --decrypt test_file.gpg without output file will print all the decrypted > > contents on the screen, which may be the reason why it takes so long. > > Generally speaking, I wouldn't consider printing to the screen "very > expensive" (i.e. print if you need to), but if you need to output a lot > of text, the other side (the terminal) will need to process and draw it > into the screen (think on it as a pipe), which will be slow with lots > of text or extremely long lines. Moreover, in Windows it will be > processed to convert LF into CRLF, and then moved into the Terminal > subsystem. You are right, my reply might be a bit misleading, what really takes time is the operation of drawing content to the terminal, if the application tries to print but you don't let it display on the screen (e.g. redirecting output to a file or /dev/null), the time consumption will not be a problem. openpgp-digital-signature.asc Description: PGP signature ___ Gnupg-users mailing list Gnupg-users@gnupg.org https://lists.gnupg.org/mailman/listinfo/gnupg-users
Multithreading with GPGME Python Bindings
Hi, I want to use the GPGME Python bindings in a concurrent way. In the documentation of the Python bindings at http://files.au.adversary.org/crypto/gpgme-python-howto.html I find no mention of multithreading. In the GPGME documentation at https://www.gnupg.org/documentation/manuals/gpgme/Multi_002dThreading.html#Multi_002dThreading I find the note, that I need to call "gpgme_check_version" before doing any multithreaded operations. Then again I find only this section about version checking in the Python binding's documentation: http://files.au.adversary.org/crypto/gpgme-python-howto.html#gpgme-version-check In this section the version check is done via a subprocess call to "gpgme-config --version". Is the call to "gpgme_check_version" maybe done implicitly by the Python bindings? Do I have to call "gpgme-config --version" before to safely do multithreaded operations with the Python bindings? Thank you for any insights, Jan ___ Gnupg-users mailing list Gnupg-users@gnupg.org https://lists.gnupg.org/mailman/listinfo/gnupg-users
