Mail Message Format

Author: Bill Heelan
$Date: 2000/07/20 23:37:55 $
$Revision: 1.1 $

This document describes the format (and some of the processing) used by the new mail system, Cacophonix.

We assume that the mail system shares a secret key with each nym (see ../../frmailauth/doc/nym-mac-key.html). This secret is generated with the nyms authentication certificate (see ../../frmailauth/doc/mail-cert.html). It's used for authentication purposes (i.e. a MAC key).

Definitions

Following RFC 822, a mail header consists of two parts: the field-name, and the field-body. For example, in

X-Freedom-Sender: bob@freedom.net

X-Freedom-Sender is the field-name, and bob@freedom.net is the field-body.

Assume the user's mail user agent (MUA) produces a message like the following

From: Nsnd
To: Nrcvt,Ircvt
Cc: Nrcvc,Ircvc
Bcc: Nrcvb,Ircvb

text

where N represents a nym, and I is a non-Freedom (Internet) user. So Nrcvc would be the nym recipient from the Cc header. Call this Morig, the original message, which includes both the RFC 822 headers and the text.

Client Side

This section describes the format of the message created by the SMTP proxy and the FMG, as well as some of the processing steps. The FMG acts as a proxy client for non-Freedom users.

Common Steps

These steps are followed by both the SMTP proxy and the FMG, with only minor differences.

  1. Sanitize the headers on the original message. To prevent possible attacks, both the client and the FMG ensure that there are no recognized X-Freedom-* headers. The client also will remove any other headers that might identify the actual sender. Call the resulting sanitized message, Msan. A more detailed explanation of header sanitizing can be found elsewhere (http://add-link-when-available).
  2. Break up the recipients into the following classes This classification is necessary because the format of the message differs according to whether the recipient is a nym or not (mail to nyms needs to be publicly encrypted with their key; mail to an Internet user doesn't), and because non-Bcc recipients must not be aware of Bcc recipients.

    Note: the FMG will not deliver mail to non-Freedom recpients.
  3. For each class of recipient make a copy of Msan.
  4. For each class of recipient generate a list of their names for the recipients header of the message envelope.

{Nym, Internet} to Nym

This section describes the format of the message destined to a nym, either from a non-Freedom user (via the FMG), or from another nym. It is assumed that the Common Steps, listed above, have been followed by the client or FMG, and that we now have a copy of Msan.

  1. Create a special binary header, Hmsg, containing the following fields
    sender [ + internet email address ]
    recipient
    hash(Msan)
    flag
    {SNsnd | MACNsnd}(hash(sender, recipient, hash(Msan)))
    From Internet to Nym, the sender field will be the FMG and there will be an internet email address field based on the message envelope sender (which will be used for the blocking list). From Nym to Nym, the client should ensure that the sender field is the sending nym's name. Since the mail is destined to a nym, there can be only one recipient. The flag field indicates whether the message is authenticated with a MAC (value is 0) or a signature (value is 1). When a nym sends mail to another nym it will use a signature, while the FMG will generate a MAC using the secret shared with the nym. (In the case of the FMG sending mail to a nym, we take advantage of the shared secret to avoid doing a public key operation.)

    Note: we want the capability of sending certs with email. We must either add a field, or use the frSec signature routines, which include space for extra data.
  2. For the pair (Hmsg, Msan), publicly encrypt Hmsg with the recipient nym's key, then using the same randomly generated symmetric key, encrypt Msan. (To publicly encrypt a large amount of data we generate a random symmetric key, encrypt the data with it, then publicly encrypt the key.)

    Note1: upon receipt of the message, we want to be able to download just the publicly encrypted Hmsg, then be able to resume symmetric decryption of the message, should we decide to download it too. This trick will save a public encryption on the FMG, and a public decryption on the client.

    Note2: the FMG must publicly encrypt the message here, rather than symmetrically encrypt using the shared secret. The latter would be vulnerable to a subpoena attack against the mail system.
  3. Only if a nym is sending the mail (i.e. not the FMG), the client will pre-generate a bounce message, Mbounce that will be delivered to the sender in case of an error. This is necessary in case an error occurs after the client has disconnected. The bounce message has a header, Hbounce, that is similar to Hmsg, with the exception that the sender and recipient fields are the sending nym's name. The bounce header and message should be publicly encrypted with the sending nym's key, rather than symmetrically encrypted with the shared secret, which would be vulnerable to a subpoena attack.
  4. Compose all these pieces into a MIME message
    X-Freedom-Message-Auth: base64(Hmsg)

    base64(Msan)

    base64(B)
    where B is
    X-Freedom-Message-Auth: base64(Hbounce)

    base64(Mbounce)
    If the message cannot be delivered, a bounce section will be base-64 decoded and put in the sending nym's mailbox, and the rest of the message will be thrown away. If the message can be delivered, the bounce section will be removed.

    Note: the exact encoding of the message has yet to be determined. It may use standard MIME boundary separators, or it may just use a character that cannot appear in base-64 encoding.

Nym to {Internet, Newsgroup}

This section describes the procedure used by the client to generate mail for a non-Freedom user, including a newsgroup. It is assumed that the Common Steps, listed above, have been followed, and that we now have a copy of Msan.

Nym to Internet

  1. Create an X-Freedom-Recipients header containing only the envelope recipients, then add it to the headers in Msan. The field-body will be a standard, comma separated list of email addresses.
  2. Create an X-Freedom-Sender header containing only the envelope sender, which should be the nym name, then add it to the headers in Msan. The field-body will have the form nym-name@freedom.net (or nym-name@trial.freedom.net, as appropriate).
  3. Create an X-Freedom-Envelope-Sig header containing a base-64 encoded signature over the field-bodies of the above two headers. Add this header to Msan. This signature allows the abuse department to validate headers.

Nym to Newsgroups

The intention is to handle news posting via qmail. The process will be similar to sending mail to an Internet user, with the exception that the envelope header will have one or more of

group-name@news.freedom.net
Qmail will interpret a domain of news.freedom.net as a request for special processing by a mail to news gateway program.

Note: need more detail.

Server Side Processing

This section describes processing done by the mail system, either when receiving mail from a nym, or when receiving mail from the Internet.

Mail From a Client

The client will communicate with the mail system using the ESMTP protocol. The mechanism by which a nym authenticates itself to the mail system is described in ../../frmailauth/doc/mail-cert.html.

  1. The mail system must check that the envelope sender matches the authenticated nym.
  2. The mail system must check that the envelope recipients conform to the classification described in Common Steps. That is, we must reject recipients containing a mix of nyms and non-nyms. We also reject multiple nym recipients.

Nym Recipient

  1. The message must contain an X-Freedom-Message-Auth header,

The mail is passed on for further processing.

Non-nym Recipient

  1. The envelope sender and X-Freedom-Sender must be the same, and must match the authenticated nym.
  2. The field-body of the message's From header is replaced by that of the X-Freedom-Sender.
  3. The message must contain X-Freedom-Recipients, X-Freedom-Sender and X-Freedom-Envelope-Sig headers.
  4. The envelope recipients must match the field-body of the X-Freedom-Recipients header.
  5. Create an X-Freedom-Mail-MAC header, whose field-body is
    date, MAC(date, fbodies)
    where fbodies is the field-bodies of the X-Freedom-Sender and X-Freedom-Recipients headers, and date is the date at which the mail is processed. The MAC verfies that the mail was processed by the Freedom system, while the date may be of use to the abuse team.

    Note: we recognize that using a MAC instead of a signature denies recipients the ability to verify, on their own, that the mail was processed by the Freedom system.

The mail is passed on for further processing.

Note: the X-Freedom-Orig-* and X-Freedom-Sig headers are deprecated in the new mail system.

Implementation Notes

  1. Where a key is used, ensure that we can distinguish between different versions of that key. This includes both signature and MAC keys.
  2. All collections of data, that may be subject to change in future versions, should have version and type fields. (E.g. Hmsg.)
  3. It is not clear whether or not we have the processing power to verify header signatures on nym to non-nym mail. In practice we could do full signature checking, random spot checks, or just verify particular instances when we receive abuse complaints.

To Do

...

Questions

  1. Even though they're text, should we base-64 encode the field-bodies of the new X-Freedom-* headers? (This would help prevent possible corruption of the information.)
  2. Should envelope senders contain full email addresses? (I.e. to distinguish nym-name@freedom.net from nym-name@trial.freedom.net.)
  3. When processing mail from a nym to a non-nym, how does the system check whether the nym's address (e.g. in X-Freedom-Sender, or envelope sender) should be nym-name@freedom.net, or nym-name@trial.freedom.net, etc.
  4. Is there any benefit of having the FMG generate an X-Freedom-Recipients header for a nym recipient?
    Short answer, no. Get details from AdamB or Roger.
  5. Is the intention to be able to verify the signature on the header, Hsan, without having downloaded the body? This isn't possible because the signature covers a hash of Msan.
    We include the hash in the header so that the signature can be verified. The client can check that the message came from the claimed nym, they just have to wait until they download the message body before that can be validated.


Copyright © 2000 Zero-Knowledge Systems Inc.
All Rights Reserved