At Linux.Conf.Au this year, I went to the keysigning party, to collect some more signatures for my PGP keys. The keysigning party loosely followed the Zimmermann-Sassamen keysigning protocol, using pre-printed lists of fingerprints, but having each person read out the fingerprint while showing their ID for verification. (Normally the Zimmermann-Sassamen keysigning protocol would have everyone check a master hash of the list, which was shown and the keysigning party and have everyone confirm their master hash matched and their details in the list matched. But the key submission period here was too long to allow many people to print their own lists at home -- submissions closed well after most people had left home.)

Now that I am back home with my checklist of validated PGP fingerprints I wanted to process the keys I checked and send out the signatures to the recipients. The LCA2015 keysigning party recommended caff to signing the keys afterwards, which is part of the signing-party package on Debian and MacPorts. So I attempted to use that for signing the keys.

caff installation

Basic installation is easy:

sudo apt-get install signing-party       # Debian, Ubuntu
sudo port install signing-party          # MacPorts

but actually getting it to function well on a modern Mac OS X system is non-trivial for various reasons. Along the way I filed a MacPorts bug on caff -- after a couple of hours of debugging "works for me" style perl -- only to later realise that the problem was a side effect of (a) another problem and (b) the error messages being hidden (by stderr redirection which seemed to be the original problem). But I am getting ahead of myself.

Because signing a bunch of keys (from a keysigning party) requires using your PGP keys a lot, most likely you will want to use gpg-agent to hold temporarily unlocked versions of the keys while you go through the signing process. Otherwise you will have to type the passphrase on your keys a lot. Installation is:

sudo apt-get install gnupg-agent         # Debian, Ubuntu
sudo port install gpg-agent              # MacPorts

On MacPorts, at least by default, it will also install pinentry-mac, a GUI tool to accept your passphrase, that pops up a always foreground window that appears on every virtual desktop until you enter your passphrase... annoying, but the other options are not that much better.

Often gpg-agent is run at startup, and keeps the keys (until some timeout) through all sessions -- and MacPorts installs launchd files that will permit that, which are disabled by default (and prints instructions on how to enable them). But it is also possible to run gpg-agent in a somewhat ad-hoc fashion, if you are using something like caff in a single terminal window -- ie single environment variable context.

To use gpg-agent in an ad-hoc manner, start a terminal window for the keysigning and then do:

GPG_TTY=$(tty)
export GPG_TTY
eval $(gpg-agent --daemon)

That will set GPG_TTY to something like /dev/ttys000 (which should match the terminal shown in who am i), and GPG_AGENT_INFO to something like /tmp/gpg-Ti9zgX/S.gpg-agent:2950:1, which is a three part (colon separated) string of (1) path to the unix domain socket for communication, PID, and agent protocol).

Unfortuantely for some reason, at least on OS X with MacPorts, it appears gpg or some other tool is unable to understand that the GPG_AGENT_INFO contains multiple parts, so you get errors like:

can't connect to `/tmp/gpg-Ti9zgX/S.gpg-agent:2950:1': No such file or directory
gpg: can't connect to `/tmp/gpg-Ti9zgX/S.gpg-agent:2950:1': connect failed

and the agent cannot be used :-( I never did figure out why the apparently documented layout did not work, but it can be made to work by throwing away the last two parts with:

GPG_AGENT_INFO=$(echo $GPG_AGENT_INFO | cut -f 1 -d :)

At this point, gpg is ready to use gpg-agent to handle the passphrase prompting for you, so we can move on to setting up caff.

However do note that it is vital that you set GPG_TTY, otherwise [certain actions that caff attempts to perform will be failed by gpg] (https://trac.macports.org/ticket/46601) with the error messages involved also being hidden. In particular certain commands seem to result in gpg reporting:

GPG_TTY must be set in shell (cannot determine automatically)

and then faililng. And caff runs gpg in such a way that those error messages are either (a) completely thrown away (eg, stderr redirected to /dev/null, or (b) only shown if you enable all the debug and tracing options (by editing the caff source and uncommenting the output statements in debug(), trace() and trace2().

Amongst other problems this also means that some commands caff uses to locate keys fail, resulting in:

[WARN] No public keys found with list-key E4D3E863 (note that caff uses its own keyring in /Users/ewen/.caff/gnupghome).
[NOTICE] No keys to sign found

even though the keys do actually exist. It took a long time to figure out that not seting GPG_TTY was the root cause of this, in part due to caff redirecting stderr to /dev/null (thanks!) -- and not redirecting stderr appearing to solve that specific problem (but not others). (More general problem reported to Debian, who seem to be the upstream.)

So do not forget to set GPG_TTY!

caff configuration

caff is configured from ~/.caffrc, and by default will use ~/.caff/ as a working directory, maintaining a completely separate gpg home directory (~/.caff/gnupghome) as well as exported copies of signed keys in ~/.caff/keys. There is an example ~/.caffrc in the documentation, but just running caff without doing anything will also write a templated file out.

The reason for using a separate gpg home directory is never really explained, but as best I can determine the idea is (a) not to fill up your main keyring with lots of keys you are mostly not using, and (b) to avoid including keys/signatures on your main keyring until the recipient has received them and added them (proving they do also have the email address listed).

This does mean that if you want to sign keys which (a) already exist in your main keyring, and (b) do not exist on a keyserver (or (c) you do not want to download the keys from the keyserver) you will have to run caff with an additional argument: --keys-from-gnupg to get encourage caff to get the keys from your main keyring.

The ~/.caffrc is a perl snippet, which is mostly supposed to set values in the $CONFIG hash. This has the advantage that you can use the power of perl in configuring things, even if it is not the most readable configuration syntax.

The mandatory settings are:

$CONFIG{'owner'}       = 'MY NAME';
$CONFIG{'email'}       = 'MY PRIMARY EMAIL';
$CONFIG{'keyid'}       = [ qw{KEYID KEYID ... } ];

where those KEYIDs are 16-character keyids rather than the 8-character ones typically printed on keysigning sheets. It appears the only way to find them is:

gpg --with-colons --list-key REGULAR_KEY_ID | awk -F : '/^pub/ { print $5; }

(ie, the 5th column in the colon separated output); they do not seem to appear anywhere else in the right format. (It is never explained why those longer keyids are used either, but I assume it is a combination of (a) being more sure of being unique and (b) being what that programmatically parsable output uses.)

The $CONFIG{'keyid'} value is used to determine which signatures are your ones and thus should be exported and sent to the recipient. If you have more than one active key (eg, "work" and "personal") and you want to create signatures with both, you will probably also want to set:

$CONFIG{'local-user'}  = [ qw{KEYID KEYID ... } ];

and you may also want to ensure the emails sent out are encrypted so those keys can read them (if only to help with debugging), by setting:

$CONFIG{'also-encrypt-to'} = [ qw{KEYID KEYID ... } ];

(Those lists can all be the same, or local-user can be any strict subset of the keyid list.)

If you already have the keys you want to sign and do not want caff trying to download them, then you will want to set:

$CONFIG{'no-download'}    = 1;

(and then make sure you use caff --keys-from-gnupg to pick up the keys you already have).

Finally you may want to set:

$CONFIG{'mail'}= 'ask-no';

so that the default is not to email the signatures, at least while you are testing (the built in default is ask-yes -- ie, ask, but default to sending email).

Sending email from caff

Sending email from caff is problematic for several reasons, but on modern desktops/laptops the most problematic is that the system is probably not a full featured Internet mail server itself. For sending email, caff just punts to the perl mail modules (especially MIME::Entity, and Mail::Mailer), which would still like to assume that sending email is as simple as running sendmail on the system and magic happens. Alas this is no longer the case for many systems.

Your options at this point are either (a) to set up at least a minimal mail server on your laptop/desktop which has some way to get out to a "real" mail server (a smarthost setup), or (b) try to persuade Mail::Mailer to do something more modern.

Modern MUA (Mail User Agent) behaviour is to connect to an outgoing mail server on TCP/587 (Mail Submission), or perhaps TCP/25 (SMTP) or TCP/465 (SMTPS), and then do SMTP AUTH to gain permission to send mail out. For security reasons SMTP AUTH is usually only permitted over a TLS (encrypted) connection. So getting Mail::Mailer to do something modern requires using TLS and SMTP AUTH. It also requires being able to verify the TLS certificate, because TLS certificates are finally being verfied by common programs (in the case of IO::Socket::SSL, this is SSL_Verify_Mode defaulting to on -- and Net::SMTP and Net::SMTP::SSL not having an option to override that -- nor any option to pass in CA-related parameters :-().

Unfortunately Mail::Mailer and friends do not support the Mail Submission (TCP/587) port, nor STARTTLS, so some kludges are required. It appears that something like Net::SSLGlue::SMTP might be a useful module for something trying to do better, as it does support STARTTLS -- but that does not seem to have been adopted.

After much experimentation I found it is possible to make Mail::Mailer behave close enough to a modern MUA by:

  • Ensuring that the SMTPS service is enabled on the mail server (listening on TCP/465, in always-TLS mode, rather than needing STARTTLS)

  • Using the smtps mail type, so that an encrypted connection is established and thus SMTP AUTH is offered.

  • Setting the SSL_CERT_FILE environment variable to point at the CA certificate used by the mail server, if it is not a widely recognised CA (eg, an organisation-wide CA); conveniently this can be done in ~/.caffrc due to it being a perl fragment, with:

    # Override SSL certificate location
$ENV{SSL_CERT_FILE} = $CONFIG{'caffhome'} .  '/naos.co.nz-ca.crt';

  • Passing authentication username/password arguments to Mail::Mailer with a username and password:

    Auth, ['AUTHUSER', 'AUTHPASS' ],

    (unfortunately this requires hard coding the password in the config file; but being a perl fragment you could possibly use a module to, eg, get it from the ~/.netrc file or a password store)

  • The From option is probably also useful, as the auto-guessed from address is unlikely to be widely recognised:

    From, $CONFIG{'email'},

    and you probably want to enable debugging until it is all sorted out with:

    Debug, 1

The full config snippet looks like:

$CONFIG{'mailer-send'} = [ 'smtps', Server, 'MAILRELAY',
                                    From,   $CONFIG{'email'},
                                    Auth,   ['AUTHUSER', 'AUTHPASS' ],
                                    Debug,  1 ];

# Override SSL certificate location
$ENV{SSL_CERT_FILE} = $CONFIG{'caffhome'} .  '/naos.co.nz-ca.crt';

which should work as it is treated as an array where the first argument is the type, and the remainder are key/value pairs (typically one would use the equivalent syntax sugar of the equals-greater-than arrow between the key and value for readability -- but the characters make the blog/RSS feeds sad). See the caff manpage for examples.

You may also want to set:

  • $CONFIG{'bcc'} = $CONFIG{'email'}; (or similar)

    But (a) the email are saved in ~/.caff/keys/DATE anyway, and (b) something in the Mail::Header is causing it to end up BCC which Mail::Mailer does not recognise as a destination address since it looks for Bcc as a literal key value, so the Bcc does not happen without patches

    As a work around one can edit the Mail::Mailer source in who_to(), and add a "BCC" variant of the "Bcc" entry, and that causes the Bcc address to be recognised... but the BCC header is still in the email so it is not a very blind CC...

    Since the messages are already being saved in a file, and the resulting messages are not actually very readable (see below), there is probably limited benefit in the Bcc option, except when trying to validate that email sending is actually working.

  • $CONFIG{'mail-template'} = ... (but due to the way that caff constructs the email, creating a multi-part MIME message with that content as one part, the signed key as another part, and then encrypting it all, the text is unlikely to be readable to anyone who does not already know what to do -- so using the built in default template is probably as good as anything).

I am not particularly fond of the email formatting (in particular it causes Thunderbird just to display a message indicating that you have to install the OpenPGP plugin), but it is "standard" so at least your emails will be no worse than anyone else's... and not as messy as, eg, choosing to PGP encrypt the key, and then clearsign the message (resulting in the PGP encrypted message being mangled such that you have to do gpg --decyrpt twice in a pipeline to be able to unpack the signed key :-( ).

Usage

Having done all of that, in theory you are now ready to actually sign some keys. With your keysigning party check sheet close at hand, and your environment and mail server setup ready to go, run:

caff --keys-from-gnupg -R KEYID

where KEYID is a key you would like to sign, that you already have (or can be downloaded if you did not configure caff to skip downloading).

It will run gpg repeatedly on your behalf to check the key, and do the keysigning steps -- either sign everything, or let you pick key uids to sign. If you have more than one key in local-user, then it will run gpg once to sign with each key. You will need your key unlocked for each time you want to sign something, but by using gpg-agent the number of actual prompts will be limited as the unlock credentials will be cached for a short period. After signing with each key, enter "save" at the gpg prompt to save the signatures and move on to the next step.

After the signing stage is complete, unless mail is set to no it will look for "recent" (defaults to "today") signatures, and either automatically send them or offer to send them. If you choose to send them, it will extract your signatures (ie signatures by one of the keys listed in $CONFIG{keyid}) and send them individually to each email address (with the idea that the recipient only gets the benefit of the keys for addresses where they have working email reception).

Once you get it working caff makes it reasonably quick to process the key checklist from a keysigning party. Unfortunately its implementation is pretty fragile, and it makes a lot of undocumented assumptions (and goes out of its way to hide the error messages so it is not obvious when its assumptions are not being met). Hopefully the above will help others avoid tripping over some of those unmet assumptions.

All of which is a reminder that GPG is "damn near unusable" :-(

Update, 2015-01-29: After some more debugging it turns out the GPG_TTY-not-set issue was self-inflicted, ironically by a wrapper script I wrote years ago that attempted to auto-set GPG_TTY from stdin or stderr -- and failed with the error I was seeing if stdin and stderr were redirected and GPG_TTY was not set. So that Macports bug got (justifiably) marked invalid. For now I've moved the wrapper aside. But I think with the benefit of hindsight the wrapper should probably have run the gpg command anyway, even if it could not set GPG_TTY sensibly, since that at least had some chance of working (lots of gpg actions do not actually need the GPG_TTY set -- just those that maniuplate the private keys) whereas exiting with an error when GPG_TTY was not set is always going to hard fail. Thus exchanging one obscure error for another obscure error. (And in the meantime it sounds like gpg itself has gone out of its way to make the errors less obscure in the GPG_TTY-needed-but-not-set case.)