Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Protocol conformance for custom mail servers

Looking up a mailbox notes that a domain’s MX servers must “support the email program protocol.” This page spells out exactly what that means for anyone weighing a from-scratch server, or an existing open-source MTA, against simply running sithbitd — and draws a line the rest of the docs don’t draw explicitly: decentralized storage is an optional layer on top of the protocol, not a requirement of it.

What conformance actually requires

Nothing on-chain checks which software sent a transaction — only whether it’s a valid one. A server “supports the protocol” if it does all of the following, regardless of implementation language or storage choice:

  • Builds and submits valid MailInstructions. The account layouts, PDA seeds, and borsh encoding are defined once in mail_model and solana_common/program_common (see the Program & PDA reference) — any server deriving the same PDAs and encoding the same instructions participates in the economic model identically to the reference implementation.
  • Checks postage before accepting mail. A sender’s frombox stamp balance must be checked (and decremented on accept) via the mail_api gRPC service or direct RPC — this is what prices out spam; see Fromboxes.
  • Seals the body to the recipient before it ever leaves the accepting server. Mail is encrypted with crypto_box_seal to the recipient’s wallet or their published delegated key — see How sealed-box encryption works.
  • Stores the sealed body somewhere reachable by the CID recorded on-chain. The Email account’s cid field is an opaque byte string as far as mail_program is concerned — the program never touches IPFS and never validates that a cid corresponds to real, distributed content (see the next section).
  • Authenticates IMAP/ POP/SMTP sessions. Either wallet-signature SASL PLAIN (verified against the connecting address, no stored secret) or a stored/sealed mail password for clients limited to CRAM-MD5/APOP — see The mail password for why both paths exist.
  • Signs outbound mail with DKIM and checks SPF/DMARC on relayed inbound mail. The chain trusts a domain’s active authority completely for relayed mail; verifying the real sender is the operator’s job — see Trust assumptions and threat model. The reference server implements full DMARC (RFC 7489) evaluation and disposition — alignment folding to organizational domains, p=reject bounces, p=quarantine → the recipient’s Junk folder, and pct sampling — selectable via sender_auth = "dmarc". It also emits RFC 7489 §7.2.1 aggregate (rua) reports (gzip XML, one per policy domain per interval, with the §7.1 external-destination check enforced on every rua target) when aggregate reporting is enabled. It also emits RFC 7489 §7.3 failure/forensic (ruf) reports when forensic reporting is enabled — one RFC 5965 ARF message/feedback-report per DMARC failure, sent direct and best-effort to the domain’s ruf addresses, headers-only by default (text/rfc822-headers) with the full message opt-in, and the same §7.1 external-destination gate applied to every ruf target (a target that fails the live-DNS authorization check declines the report). The receiving side — ingesting other operators’ aggregate reports about your own domains — is an optional operator feature, not a conformance requirement; see RFC 7489 §7.2 ingestion below.

None of this requires joining a peer-to-peer network. It requires chain-ABI conformance, the sealed-box crypto, and some addressable place to put ciphertext.

Decentralization is optional, not required

The cid field’s name suggests IPFS, and the reference implementation does use real content identifiers — but two things are worth being precise about:

  1. The trustless read path doesn’t re-verify the hash either. The client-side reader (webclients/shared/trustless.js) fetches {gateway}/ipfs/{cid} and unseals whatever comes back; it does not recompute the CID’s multihash and compare it to the fetched bytes. The tamper-evidence in practice comes from the sealed-box authenticated encryption, not from CID verification: swap or corrupt the bytes behind a CID and the recipient’s decrypt fails loudly, whether or not anything ever checked the hash. A CID-shaped identifier resolved through any addressable store — not necessarily a distributed IPFS swarm — gives the recipient the same cryptographic guarantee.
  2. The reference implementation itself defaults to no swarm. sithbitd’s embedded IPFS node (ipfs_daemon/ipfs_swarm) ships with swarm = None — no libp2p, no Kademlia DHT, no bitswap — unless an operator explicitly configures [swarm] with public listen addresses and provide = true (see sithbit-ipfsd). Out of the box, sithbitd already runs in exactly the mode this page is describing: chain economics fully live, bodies content-addressed and servable over a private HTTP gateway, with zero participation in the public IPFS network.

So a server that stores sealed bodies in a conventional store (a database, a filesystem, S3) behind its own GET /ipfs/{cid}-shaped endpoint, without ever joining the public swarm, is not a lesser or non-conformant implementation — it’s the same posture the reference implementation defaults to. Real distribution (pinning to the public network, or running a shared-bucket cluster) is an enhancement you opt into, layered on top of a protocol that doesn’t require it.

What you give up by skipping real distribution, honestly stated:

  • Availability beyond your own infrastructure. If your server or its storage goes down, there is no other peer or pinning service holding a copy — unlike content actually pinned onto the public network, or handed to a pinning service (see IPFS storage: benefits to users).
  • No public discoverability. A stranger running a generic IPFS client against <cid> won’t find your privately-stored bytes — only your own gateway resolves them.
  • Integrity is unaffected. Recipients still get the same tamper-evidence either way, since it comes from the encryption, not the network.

RFC 8314: cleartext is obsolete — TLS before credentials

RFC 8314 (“Cleartext Considered Obsolete: Use of Transport Layer Security for Email Submission and Access”) requires mail submission and mail access to run over TLS, and requires a server to refuse authentication on an unprotected connection rather than inviting credentials into the clear. The reference servers enforce this by default in production posture — require_tls is on out of the box for the SMTP submission edge, IMAP, and POP — declining credentials until the connection is protected, each in its protocol-appropriate shape:

  • SMTP submission refuses AUTH (and MAIL FROM) before STARTTLS with a 530 5.7.0 Must issue a STARTTLS command first, and hides the AUTH capability from the EHLO response so clients aren’t invited to authenticate in the clear. The gate is the require_tls && !tls_active check in smtp_session/src/session/ready.rs (the auth and mail handlers), and the server default is require_tls.unwrap_or(mode == Submission) in smtp_server/src/config.rs — on for the submission edge.
  • IMAP advertises LOGINDISABLED in the pre-TLS CAPABILITY and refuses LOGIN/AUTHENTICATE with NO [PRIVACYREQUIRED] (RFC 5530) until STARTTLS completes. The gate is credentials_refusal in imap_session/src/session/not_authenticated.rs; require_tls defaults to true in imap_server/src/config.rs.
  • POP3 refuses USER/PASS before STLS with -ERR Must issue STLS command first, and discards any pre-TLS USER after the upgrade (a STARTTLS injection defense). The gate is tls_gate in pop3_proto/src/session/authorization.rs; require_tls defaults to true in pop_server/src/config.rs.

On the client side of submission, the spooler’s [spooler.smarthost] relay path can dial with implicit TLS (implicit_tls = true — the transport §3.3 prefers for submission) instead of STARTTLS — see the [spooler] reference.

These are runtime policy gates, not compile-time ones: the legacy server compiled its TLS gate out of Debug builds, and the reference servers deliberately do not. The zero-config developer stack (loopback plaintext binds) is a separate, explicitly opt-in convenience, outside this production conformance claim.

RFC 8461: MTA-STS — downgrade-resistant outbound TLS

RFC 8461 (“SMTP MTA Strict Transport Security (MTA-STS)”) lets a receiving domain declare that its MX hosts support TLS with a valid certificate, so a sending MTA can refuse to deliver over an unauthenticated or plaintext channel — closing the STARTTLS-stripping downgrade that opportunistic TLS (RFC 7435) leaves open. The relay implements the sending side, on by default (the mta_sts switch in the [spooler] reference); domain-sithbit implements the publishing side (the last bullet):

  • Policy discovery and parsing live in mail_spooler/src/mta_sts.rs: the _mta-sts.<domain> TXT probe (§3.1, with a transient/definitive split so a resolver hiccup is never mistaken for policy withdrawal), the HTTPS fetch of https://mta-sts.<domain>/.well-known/mta-sts.txt (§3.3 — 10-second timeout, redirects refused, 64 KiB body cap), the §3.2 policy parser, and the §4.1 MX-pattern matcher (exact match or a *. wildcard covering exactly one leftmost label).
  • Enforce mode is the relay’s attempt_enforced branch in mail_spooler/src/pipeline/relay.rs: only MX targets matching the policy’s mx patterns are dialed, each demanding STARTTLS with a certificate verified against the webpki roots for the MX hostname (TlsVerify::Strict in mail_spooler/src/smtp_out.rs — the same strict verifier the smarthost path uses). Any TLS failure — STARTTLS missing or refused, a handshake or certificate error — and zero matching MX targets defer the mail on the normal retry schedule (§5’s transient handling): enforce never bounces on a TLS failure and never falls back to plaintext.
  • The policy cache honors §5.1: policies are cached per domain for their max_age (clamped to one year) and keyed to the TXT record’s id for rollover, and an unexpired cached policy keeps being applied through a DNS strip or transient resolver failure — which is what defeats record-removal attacks against senders that have already seen the policy.
  • Testing mode delivers opportunistically and logs (tracing::warn!) each MX target that would fail under enforce. With [spooler.tlsrpt] enabled, each dialed attempt also records an RFC 8460 result row under its STS policy context — see TLS-RPT below — which is the feedback loop testing mode is designed to be watched through.
  • The publish side lives in domain-sithbit (domain_sithbit/src/mta_sts.rs), serving GET /.well-known/mta-sts.txt for the domains the instance fronts. The §3.2 serializer renders the policy body once at startup from the optional [mta_sts] config section — version/mode/mx/max_age, every line CRLF-terminated — and validation is fail-fast at boot: an unknown mode name, an enforce/testing policy with no mx pattern (§3.2 requires one), or a max_age above the one-year ceiling refuses to start rather than serving a policy senders would reject or silently shorten (the ceiling is fenced equal to the consume side’s clamp by a round-trip test through the spooler’s parser). With no [mta_sts] section the route answers 404 — publication is opt-in per instance, and there is no per-domain policy map: one instance serves one policy. HTTPS is the fronting proxy’s job (senders fetch https://mta-sts.<domain>/.well-known/mta-sts.txt, so the proxy needs a certificate for that hostname), no TLS-RPT (RFC 8460) reports are consumed on this side either, and the _mta-sts.<domain> discovery TXT record — with its id bump on every policy change — stays operator-managed DNS: see DNS setup.

RFC 7672: DANE — DNSSEC-pinned outbound TLS

RFC 7672 (“SMTP Security via Opportunistic DANE TLS”) lets a receiving domain pin its MX hosts’ TLS certificates in DNSSEC-signed TLSA records, closing the same STARTTLS-stripping downgrade as MTA-STS — but with DNSSEC as the trust base, so it has neither the trust-on-first-use nor the cache-lifetime residual. The relay implements the sending side, on by default (the dane switch in the [spooler] reference), preferring DANE over MTA-STS wherever both apply:

  • TLSA discovery and classification live in mail_spooler/src/dane.rs: the _25._tcp.<mx-host> lookup rides a DNSSEC-validating resolver, and every answer record’s validation proof is checked — one bogus or unsigned hop taints the whole chain (§2.2.2). The §3.1.3 usability rules apply: DANE-EE(3) and DANE-TA(2) with known selectors/matching types are usable; PKIX-TA(0)/PKIX-EE(1) and unknown registry values are not. The per-host verdict is one of: verify (usable records — pin the handshake), mandatory unauthenticated TLS (a validated RRset that is all-unusable, §2.2/§3.1.3), not applicable (no TLSA, unsigned zone, or a validated denial — the host stays on the MTA-STS/opportunistic path), or unusable (bogus validation or a failed lookup — the host is never dialed).
  • The DANE verifier (TlsVerify::Dane in mail_spooler/src/smtp_out.rs) matches the presented chain against the records: DANE-EE matches the end-entity certificate alone, with name, expiry, and chain checks all skipped (§3.1.1 — the DNSSEC-signed record is the trust statement); DANE-TA requires some presented chain certificate to match a TLSA record AND the end entity to path-validate (rustls-webpki) with that certificate as the trust anchor, expiry and the §3.2.3 server-name check enforced. Full-certificate and SPKI selectors, exact/SHA-256/SHA-512 matching.
  • Relay composition is the per-host planner in mail_spooler/src/pipeline/relay.rs (plan_hosts): DANE applies to an MX host only when the MX RRset itself validated (§2.2.1) AND that host’s TLSA chain validated with usable records — and then it outranks an MTA-STS policy, including its mx pattern filter (RFC 8461 §2). An all-unusable TLSA set demands TLS without authentication, except under an MTA-STS enforce policy, which stays the stricter floor. Any DANE failure — a handshake that matches no record, a bogus TLSA chain, every host excluded — defers the mail on the normal retry schedule, never a plaintext fallback and never a bounce.
  • MX-less domains (§2.2.1). A domain with no MX record — the implicit-A fallback, where the connect host is the domain itself — is not excluded from DANE: when the denial of MX existence is DNSSEC-proven (a Secure validation proof on the negative answer’s SOA), the fallback counts as a validated answer, and TLSA records published at _25._tcp.<domain> are consulted and enforced exactly as for an MX host. The honest subset that remains is narrower: a denial that arrives without a validatable SOA — or whose proof is insecure, indeterminate, or bogus — stays insecure and skips DANE, and unsigned MX-less zones behave exactly as before (opportunistic TLS).
  • Documented subsets. The TLSA base domain is the MX hostname as published: CNAME chains are followed (each hop must validate Secure), but the §2.2.3 alternate base-domain derivation from A/AAAA-expansion is not performed — a subset that only ever loosens toward today’s opportunistic posture, never past a published policy.
  • Observability is warn-level logging on unusable TLSA chains and DANE handshake failures — and, with [spooler.tlsrpt] enabled, every dialed attempt (and every host DANE excludes before dialing) records an RFC 8460 result row under its TLSA policy context; see TLS-RPT below.
  • Scope: sending side only. SithBit does not generate TLSA records for its own domains; an operator who wants inbound protection publishes them in their DNSSEC-signed zone — see DNS setup.

RFC 8460: TLS-RPT — SMTP TLS reporting

RFC 8460 (“SMTP TLS Reporting”) is the feedback loop for the two mechanisms above: a sending MTA records how its outbound TLS sessions actually went — per recipient domain, per governing policy — and delivers a daily aggregate report to whatever addresses that domain names in its _smtp._tls TLSRPT record, so the domain’s operator sees downgrade attempts and misconfigured MX hosts from the senders’ vantage point. The reference server implements the sending side — recording and reporting — behind the single [spooler.tlsrpt] switch, off by default (the configuration reference):

  • Result recording happens in the relay’s per-host attempt loop (mail_spooler/src/pipeline/tlsrpt.rs), direct-to-MX path only: each dialed attempt that produced TLS evidence lands one row carrying the RFC 8460 §4.2 policy context that governed it — tlsa (the verified TLSA records rendered as policy strings), sts (the MTA-STS policy body), or no-policy-found — and either a success tally or a §4.4 failure-details block. Hosts a policy excludes before dialing land never-dialed failure rows too: a DANE-unusable host (excluded at the resolver/proof level, no TLSA records assessed) records dnssec-invalid with a bare tlsa policy block, and an MX target outside an enforce-mode MTA-STS policy records sts-policy-invalid rendering the enforce policy body — the planner’s diagnostic rides failure-reason-code. Retries re-record, and identical rows aggregate by failed-session-count at fold time. Recording is strictly observational: a recorded failure still defers/retries exactly as the enforce sections above describe, and a failed row write warns without ever changing a delivery outcome. Smarthost mode records nothing — a smarthost’s TLS posture is not the recipient domain’s.
  • The report worker (mail_spooler/src/pipeline/tlsrpt_report.rs) runs on the same switch: at each interval_hours tick it folds the pending rows into one RFC 8460 report per recipient domain, discovers the domain’s rua= targets from its _smtp._tls.<domain> TXT record, and delivers over both channels — mailto: targets ride the normal outbound relay, DKIM-signed on spool entry (so the configured email must be a local, DKIM-signable address; its domain doubles as the report’s submitter identity), and https: targets receive the gzip-compressed JSON directly as an application/tlsrpt+gzip POST (10-second timeout, redirects refused — the MTA-STS fetcher’s settings). Unlike DMARC aggregate reporting there is no §7.1-style external-destination authorization gate: RFC 8460 defines none, so a report goes wherever the published record points.
  • Delivery bookkeeping, stated honestly. Rows are deleted only after a domain’s report reached every target, so a crash between send and delete — or a partial multi-target failure, which defers the whole domain — can re-deliver the window; the deterministic report-id (<end-time>_<domain>) lets receivers de-duplicate. A domain that definitively publishes no TLSRPT record (or one naming no rua= target) has its rows dropped rather than pinned forever; only transient DNS or delivery failures keep rows pending for the next tick. An unparseable pending row is poison: warned and deleted.
  • Recording gaps, on record. The rustls seam collapses the RFC 8460 certificate result taxonomy (certificate-expired, certificate-host-mismatch, …) into the general validation-failure code, with the raw TLS error detail preserved in failure-reason-code. Success rows are flag-truthful — recorded only when the outcome says the conversation actually ended on TLS — so a completed plaintext opportunistic session (TLS never negotiated, including a declined STARTTLS offer that continued in the clear) records no row at all: neither a §4.1 TLS session nor a failed attempt. The honest limit that remains: the seam cannot distinguish “STARTTLS never offered” from “offered but declined, continued plaintext” — both go unrecorded, with starttls-not-supported failure rows reserved for enforced postures that abort. Unreachable or timed-out hosts still record nothing, and MTA-STS testing-mode mismatches are warn-logged, never recorded.
  • No receiving side. Ingesting other operators’ TLS reports about your own domains is not implemented — inbound reports are ordinary delivered mail (there is no TLS-RPT sibling of the DMARC rua ingestion below). To request reports about a domain you operate, publish the TLSRPT record — see DNS setup.

RFC 7489 §7.2: ingesting DMARC aggregate reports

The emitting side of DMARC reporting is covered above; the reference server also implements the receiving side of RFC 7489 §7.2 — what happens when another operator’s receiver mails an aggregate (rua) report to a domain you operate. The posture is deliberately ingest, store, and surface — nothing more:

  • Reaching the mailbox at all. An external reporter never holds a prefunded frombox, so the postage gate would refuse it like any other stranger. The [smtp] postmaster_wallet setting implements the RFC 5321 §4.5.1 postmaster exemption in the SMTP driver (smtp_server/src/driver.rs): RCPT to bare postmaster or postmaster@<local-domain> (case-insensitive, per §4.5.1) skips alias resolution and the frombox/postage check entirely and delivers to the configured wallet — a foreign-domain postmaster stays a relay request. Unset (the default), refusals are byte-identical to the unconfigured behavior.
  • Parse and store (mail_spooler/src/adapters/dmarc_rua.rs). With [spooler.dmarc_rua_ingest] enabled, delivery to a matched local recipient (a bare local-part entry matches at any local domain, a full address exactly; default ["postmaster"]) parses the raw message with mail-auth’s RFC 7489 parser — MIME wrapping and gzip/zip report bodies handled — and stores the parsed report, serialized verbatim as JSON, at blob key dmarc_rua/<id>.json. The id is org_name!report_id!begin!end per the §7.2.1.1 filename convention, sanitized to [A-Za-z0-9._-] (every other byte, including the ! separators, becomes _; 200-byte cap). Ingestion is strictly additive: the message still delivers to the mailbox normally, a malformed report warns and delivers, redelivery overwrites the same key idempotently, and relay recipients never trigger it.
  • Surface. The account API’s admin routes (GET /v1/admin/dmarc-reports list, GET /v1/admin/dmarc-reports/{id} fetch) read the stored JSON back — see account-api, and DNS setup for wiring the rua= record to your own deployment.

What is deliberately not implemented, so an operator knows what this feature is not:

  • No automated disposition. Auto-disabling or suspending accounts from RUA data was considered and rejected as a category error: aggregate-report rows carry no join key back to local wallets, and the rows failing your policy are almost always third-party spoofers, not your users. The reports exist for a human operator to read.
  • No ruf ingestion. Failure/forensic reports are emitted (above) but not consumed — inbound ARF messages are ordinary mail.
  • No pruning. Nothing deletes stored reports yet; they accumulate under the dmarc_rua/ blob prefix until an operator clears them — a documented limitation, like the TLS-RPT gaps above.

Self-authenticating TLS (optional, for DHT discovery)

Everything above is what a server must do to participate in the economic protocol. A server that additionally wants to be discovered over the DHT (rather than published in DNS SRV/A records) opts into one more behavior — self-authenticating TLS, the connection half of decentralized service discovery:

  • The node presents a self-signed certificate whose key is its delegated node key. The certificate is not chained to a public CA. It carries the node’s authority-signed SignedDelegation in a custom X.509 v3 extension under OID 1.3.6.1.4.1.58888.1.1 (an unregistered placeholder enterprise number — not IANA-registered; an independent implementation must match the constant. Registering a real PEN is a tracked external prerequisite that MUST replace this arc before mainnet). The delegation binds {domain, proto, node_pubkey, expiry} and chains to the domain’s on-chain MailDomain.authority.
  • The client verifies against the chain, not a CA or the hostname. A conforming client (SithBit’s NodeDelegationVerifier, a rustls ServerCertVerifier) extracts the delegation from the leaf certificate, resolves the domain’s authority from chain, verifies the delegation’s signature against it, and enforces that the certificate’s public key equals the delegated node_pubkey. Intermediates, the SNI/server name, and OCSP are deliberately ignored — trust flows only from the on-chain authority. This makes a node impersonation-proof even if the DHT record that pointed the client at it was poisoned: a bad address just fails the handshake’s chain check.

This is the inverse of the client-certificate SASL EXTERNAL mechanism (there the client proves a wallet identity to the server; here the server proves a delegated domain identity to the client). It is entirely optional: a server published the classic way in DNS, presenting an ordinary CA-issued certificate, is fully conformant — self-auth TLS matters only if you want the server found and trusted through the DHT with no DNS and no public CA.

Should you graft this onto an existing MTA?

Given the above, the SMTP-accept edge — envelope validation, TLS, the postage check — is the one piece with a plausible plugin surface in a mature MTA: a Postfix Milter or an Exim ACL could call the mail_api gRPC service to check and decrement stamps before accepting a message, in the same shape as existing greylisting or reputation Milters.

Everything downstream of “accepted” does not have a comparable plugin surface:

  • Local delivery must seal the body to the recipient’s key and land it in CID-addressed storage instead of a Maildir/mbox — no standard local delivery agent does this.
  • IMAP/POP retrieval must decrypt from that store on fetch — Dovecot’s storage backends assume a conventional mailbox format.
  • Wallet-signature SASL PLAIN, and the CRAM-MD5/APOP fallback that needs a server-held secret, has no drop-in mechanism in stock Cyrus SASL or Dovecot auth without custom code either way.
  • DKIM signing on spool entry, the relay retry schedule, RFC 3464 DSNs, and on-chain settlement bookkeeping are all mail_spooler’s job regardless of which SMTP edge accepted the message.

So grafting SithBit support onto Postfix/Exim/Dovecot buys you a mature MTA’s SMTP-edge tooling (anti-abuse, TLS hardening, operational familiarity) at the acceptance step, but the storage, crypto, wallet auth, and settlement layers still have to be written from scratch — which is what mail_spooler already is. sithbitd is the reference implementation and the recommended path for standing up a domain; a bespoke server is worth building primarily if keeping an existing MTA’s edge tooling matters more to you than the extra integration work, not because it’s meaningfully less total effort.

Conformance checklist

  • Build and submit valid MailInstruction/AliasInstruction transactions (mail_model, solana_common).
  • Check and decrement frombox stamps before accepting mail (mail_api gRPC).
  • Seal bodies with crypto_box_seal to the recipient’s wallet or published delegated key.
  • Store sealed bodies under a content address reachable at the URL you publish on-chain — real IPFS or a conventional store, your choice.
  • (optional) Pin or announce that content on the public IPFS network for availability beyond your own infrastructure.
  • Require TLS for submission and mail access, refusing credentials before the connection is protected (RFC 8314) — SMTP AUTH, IMAP LOGIN/AUTHENTICATE, POP USER/PASS.
  • Accept wallet-signature SASL PLAIN and/or a stored mail password for clients limited to legacy SASL mechanisms.
  • Sign outbound mail with DKIM; verify SPF/DMARC on relayed inbound mail.