The updated keyserver seems to have helped. I added code to log the keyserver debug output in the server logs (it appears in error.log) for later review if necessary.
On Fri, 15 Feb 2019 at 09:54, sebb <[email protected]> wrote: > It looks like Whimsy uses gpg2 (2.1.1) rather than gpg. > The gpg2 man page says: > > * verbose* > > This option has no more function since GnuPG 2.1. > Use the *dirmngr* configuration options instead. > > > *debug* This option has no more function since GnuPG 2.1. > Use the *dirmngr* configuration options instead. > > > > However I don't yet know how to pass options to dirmngr. > > > I noticed that pgpkeys.mit.edu is very slow to respond, so I tried > hkps.pool.sks-keyservers.net > > instead. That was much quicker, so I think it is worth trying it. > > On Fri, 15 Feb 2019 at 03:49, Craig Russell <[email protected]> wrote: > >> Looks like the version of gpg that workbench uses does not support >> --keyserver-options debug. >> >> Any ideas how to get more information from the failure? >> >> The man page on whimsy-vm4 would indicate that debug is a suitable >> keyserver-option. >> >> Any ideas: >> >> gpg: Signature made Fri Feb 15 01:02:20 2019 UTC using RSA key ID >> 7B061396 gpg: Can't check signature: No public key gpg: keyserver option >> 'verbose' is unknown gpg: keyserver receive failed: No data >> >> On Feb 13, 2019, at 2:56 AM, sebb <[email protected]> wrote: >> >> BTW, to distinguish stdout and stderr one can redirect them to different >> files: >> >> $ command 2>stderr.out >stdout.out >> >> [Some IDEs colour them differently] >> >> On Tue, 12 Feb 2019 at 23:55, Craig Russell <[email protected]> wrote: >> >>> Tldr; the debug output starting with "< HTTP/1.1 " >>> Seems to be the most useful. Here is some of the debug output: >>> < HTTP/1.1 502 Proxy Error >>> < HTTP/1.1 200 OK >>> < HTTP/1.1 404 Not found >>> < HTTP/1.1 503 Service Temporarily Unavailable >>> >>> >>> The 200 is a good key retrieval. The 400 is definitely key not found. >>> The 502 and 503 can be retried. >>> >>> Again, I'm keen to try adding "debug" to the --recv-keys command and see >>> what happens. In parallel, if someone can figure out how to extract just >>> the HTTP/ part of the debug output, we can programmatically distinguish >>> between the key not found and the temporary errors. >>> >>> Craig >>> >>> On Feb 12, 2019, at 3:44 PM, Craig Russell <[email protected]> wrote: >>> >>> I just read the code again and I'll copy it here: >>> >>> # run gpg verify command >>> out, err, rc = Open3.capture3 gpg, '--verify', signature.path, >>> attachment.path >>> # if key is not found, fetch and try again >>> if >>> err.include? "gpg: Can't check signature: No public key" or >>> err.include? "gpg: Can't check signature: public key not found" >>> then >>> # extract and fetch key >>> keyid = err[/[RD]SA key (ID )?(\w+)/,2].untaint >>> out2, err2, rc2 = Open3.capture3 gpg, '--keyserver', 'pgpkeys.mit.edu', >>> '--recv-keys', keyid >>> # run gpg verify command again >>> out, err, rc = Open3.capture3 gpg, '--verify', signature.path, >>> attachment.path >>> # if verify failed, concatenate fetch output >>> if rc.exitstatus != 0 >>> out += out2 >>> err += err2 >>> end >>> end >>> >>> >>> On Feb 11, 2019, at 12:25 PM, Craig Russell <[email protected]> >>> wrote: >>> >>> Hi Shane, >>> >>> On Feb 9, 2019, at 5:07 AM, Shane Curcuru <[email protected]> wrote: >>> >>> Can;'t type enough for code yet, but miss coding, so... ideas: >>> >>> When verifying zig, you want workbench to: >>> >>> - IF we get here: (i.e. there was an error gettign key) >>> >>> https://github.com/apache/whimsy/blob/master/www/secretary/workbench/views/actions/check-signature.json.rb#L40 >>> >>> >>> First, the entire error message should be displayed instead of just "no >>> data". >>> >>> >>> Actually, if we get here, there was an error verifying the signature, >>> *not* getting the key. Getting the key is done after the #extract and fetch >>> key line, using the command >>> gpg, '--keyserver', 'pgpkeys.mit.edu', >>> >>> '--recv-keys', keyid >>> The code then tries to verify the signature again, and if it fails, >>> concatenates the failure to obtain the key with the failure to verify. >>> >>> I've never seen anything except "no key data" so I believe that the real >>> problem here is that the gpg --recv-keys command just doesn't give much >>> data if it doesn't successfully receive the key. >>> >>> So perhaps all we really need to do here is to give more options on the >>> recv-keys command. Perhaps using --keyserver-options would help, with >>> options of "verbose" or "debug". >>> >>> Using verbose does not seem to help: >>> gpg --keyserver pgpkeys.mit.edu --keyserver-options verbose --recv-keys >>> 91CDDE15 >>> gpg: requesting key 91CDDE15 from hkp server pgpkeys.mit.edu >>> gpgkeys: key 91CDDE15 can't be retrieved >>> gpg: no valid OpenPGP data found. >>> gpg: Total number processed: 0 >>> gpg: keyserver communications error: keyserver helper general error >>> gpg: keyserver communications error: Invalid public key algorithm >>> gpg: keyserver receive failed: Invalid public key algorithm >>> >>> Using debug gives more information: >>> >>> bash-3.2$ gpg --keyserver pgpkeys.mit.edu --keyserver-options debug >>> --recv-keys 91CDDE15 >>> gpg: requesting key 91CDDE15 from hkp server pgpkeys.mit.edu >>> gpgkeys: curl version = libcurl/7.54.0 LibreSSL/2.0.20 zlib/1.2.11 >>> nghttp2/1.24.0 >>> * Trying 18.9.60.141... >>> * TCP_NODELAY set >>> * Connected to pgpkeys.mit.edu (18.9.60.141) port 11371 (#0) >>> > GET /pks/lookup?op=get&options=mr&search=0x91CDDE15 HTTP/1.1 >>> Host: pgpkeys.mit.edu:11371 >>> Accept: */* >>> Pragma: no-cache >>> Cache-Control: no-cache >>> >>> < HTTP/1.1 503 Service Temporarily Unavailable >>> < Date: Tue, 12 Feb 2019 23:31:23 GMT >>> < Content-Length: 323 >>> < Connection: close >>> < Content-Type: text/html; charset=iso-8859-1 >>> < >>> * Closing connection 0 >>> gpgkeys: key 91CDDE15 can't be retrieved >>> gpg: no valid OpenPGP data found. >>> gpg: Total number processed: 0 >>> gpg: keyserver communications error: keyserver helper general error >>> gpg: keyserver communications error: Invalid public key algorithm >>> gpg: keyserver receive failed: Invalid public key algorithm >>> >>> I'm not clear from the console output here which output is part of the >>> stdout and which is part of errout. But I'd be willing to just try adding >>> debug to the command and see what happens. >>> >>> WDYT? >>> >>> Craig >>> >>> >>> - And if >>> err.include? "gpg: Can't check signature: No public key" or >>> err.include? "gpg: Can't check signature: public key not found" >>> >>> >>> - Then Do: enable the action that sends pubkey.erb (so you can push the >>> button - or do you want it to automatically send that upload email)? >>> >>> >>> The action to send pub key.erb is always enabled. But the message should >>> have the command that failed, e.g. >>> http://pgpkeys.mit.edu/pks/lookup?search=91CDDE11&op=index >>> >>> That way, the sender has some idea of which key was being requested. >>> >>> >>> - ELSE: (other error like proxy/timeout) >>> - Disable the pubkey.erb >>> >>> >>> No need. >>> >>> - Allow the Secretary to re-process the check sig action. >>> >>> >>> Do you want it to simply give you the button to immediately re-chceck >>> for the key, >>> >>> >>> That would be a nice option. Just clicking on the .pdf file doesn't >>> automatically try to download the public key from what I can tell. >>> >>> or do you want to leave this whole msg and come back later? >>> >>> >>> Just having the entire error message displayed with no further >>> editing/analysis is fine. From the message, secretary can decide whether to >>> send the "upload pub key" message or not. And in fact, sometimes it takes a >>> couple of upload messages before the sender gets around to uploading the >>> key. And having the command as part of the message might allow the sender >>> to just click on the link and see if they are successful accessing their >>> own public key. >>> >>> Craig >>> >>> >>> -- >>> >>> - Shane >>> Director & Member >>> The Apache Software Foundation >>> >>> Craig Russell wrote on 2/8/19 8:18 PM: >>> >>> Hi, >>> >>> When a document has an associated .asc file, whimsy automatically tries >>> to download the public key in order to verify the signature. >>> >>> But the key server is often busy and just cannot service the request >>> timely. >>> >>> The current behavior of whimsy does not distinguish among at least three >>> cases: >>> >>> No key found >>> Timeout >>> Proxy error >>> >>> No key found is the only case where the secretary should send the >>> "upload public key" message to the submitter. >>> >>> In the other two cases, the secretary should retry (later) until a >>> definitive No key found error is received or the public key is downloaded. >>> >>> Can the actual error be extracted from the message to avoid confusion? >>> >>> Thanks, >>> >>> Craig >>> >>> Craig L Russell >>> Secretary, Apache Software Foundation >>> [email protected] http://db.apache.org/jdo >>> >>> >>> >>> Craig L Russell >>> Secretary, Apache Software Foundation >>> [email protected] http://db.apache.org/jdo >>> >>> >>> Craig L Russell >>> Secretary, Apache Software Foundation >>> [email protected] http://db.apache.org/jdo >>> >>> >>> Craig L Russell >>> Secretary, Apache Software Foundation >>> [email protected] http://db.apache.org/jdo >>> >>> >> Craig L Russell >> Secretary, Apache Software Foundation >> [email protected] http://db.apache.org/jdo >> >>
