Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

The mail-grpc gateway topology (design note)

This note records a deliberate architecture decision (2026-07-15): the mail-grpc chain gateway stays a separate service rather than being folded into the servers that consume it. It explains what the gateway actually is, who talks to it, why the alternatives were rejected, and what would reopen the question — so the “why is there a gateway?” answer survives in one place.

What mail-grpc actually is: three roles in one process

The gateway is not one thing but three, and any topology decision has to place all of them:

  1. A remote-signing write gateway. Five RPCs (SendMail, DeleteMail, RefundMail, ClaimBounty, RefundBounty) build, sign, and submit Solana transactions. Every one of them signs with a single process-wide keypair — the keypair described in the mail-grpc chapter — which acts as fee payer and sole signer. For the bounty RPCs that key is the on-chain wallet being settled: the gateway is, in effect, the operator’s on-chain mail identity.
  2. A chain-read gateway. Thirteen RPCs (ResolveAlias, GetFrombox, GetMailbox, GetMailboxKey, GetTransactionStatus, FindMessage, GetMailDomain, ListAuthoritativeDomains, BrowseListings, ListParticipants, GetSenderAttestation, GetPinLease, GetSenderReputation) read accounts and signatures directly from finalized chain state — GetPinLease as a filtered getProgramAccounts scan for a CID’s leases, GetSenderReputation as two account fetches (the wallet’s reputation PDA plus the postoffice) folded through the on-chain pricing rule. No key material is involved beyond the operator identity two of them imply (ListAuthoritativeDomains answers “which domains is my signing key authoritative for”).
  3. The only home of the off-chain alias/sales indexer. ListAliases and ListSales are served from a SQLite index the gateway builds by scanning transaction history, because alias names are not recoverable from chain state alone (accounts key on hashes). This role is an irreducible singleton: deleting the gateway would not delete a deployable role, it would relocate one — and hand every consumer that wants alias listings a new protocol to reach it.

Two keys, two services — a common conflation

The gateway’s signing key is not the postmaster’s standing delegate key, though the two are easy to conflate:

  • mail-grpc’s signing keypair is a general fee-payer/signing key for mail traffic (sends, deletes, bounty settlement). It can be loaded from Azure Key Vault.
  • The standing delegate (delegate_key_file) lives in domain-sithbit, authorizes domains on-chain, and is re-read from disk on every request — see Postmaster key custody.

Two keys, two services, two custody stories. Keeping the write gateway separate keeps the mail-signing key in exactly one process — the only AKV-capable holder — instead of copying it to every worker.

Who consumes it — and who never does

Every consumer is another server; no end-user client ever dials the gateway:

  • sithbitd is the only writer. Its chain workers drive SendMail and DeleteMail off store-backed job queues, serialized per wallet by a store lease — which is why a cloud-store fleet of many sithbitd instances can safely share one gateway. Its SMTP accept path asks the gateway for postage at RCPT time, and its at-rest sealing asks it for reader keys.
  • A standalone smtp-server MX performs read-only postage verification (ResolveAlias + GetFrombox per recipient).
  • account-api routes compose recipients (alias → postage check) and backs its /v1/chain REST proxy with gateway reads. Its balance and transaction-submit routes deliberately use its own direct Solana JSON-RPC instead — client-signed transaction relay and plain balance reads need no gateway semantics. That split is documented here as intentional; it is not drift to “fix”.
  • sithbit-console’s balances pane reads GetMailbox/GetFrombox directly.
  • Web and plugin clients never touch it. There is no gRPC-web anywhere: the webmail/marketplace/onboarding panes reach chain data through account-api’s REST proxy, and trustless webmail goes straight to a public Solana RPC endpoint with client-side decoding — bypassing the operator’s servers entirely.

The dependency contract matters as much as the call graph: consumers link only the mail_api proto crate (wire types + generated client, a handful of dependencies) instead of the full Solana host stack (~a dozen solana-* crates plus their transitive weight) that mail-grpc absorbs once. The same seam is the test seam — the protocol servers’ suites fake the SolanaMail service in-process, which is what keeps them hermetic and fast.

Network posture: private-network-only, by design

The gRPC surface carries no TLS and no authentication — anyone who can reach the port can sign with the gateway’s key. That is acceptable under exactly one posture, which is now the documented requirement:

mail-grpc must only be reachable on the private network segment where the other SithBit servers run. Bind it to loopback on a single box, or to a private interface/subnet in a fleet — never to a public address. It is server-to-server plumbing, not a public API.

The shipped deployment already leans this way (the compose chain profile serves 127.0.0.1:50051), and the iac/ templates’ opt-in mail-grpc units are the posture’s reference implementation at fleet scale: a private subnet the operator brings, no public ingress path, in both clouds. One operational caveat follows: an administrator running sithbit-console from outside that network needs a route in (VPN or bastion) for the balances pane — everything else the console does goes through account-api.

Options considered

Ops surfaceKey custodyTest seamDep graph
A. Keep separate (chosen)one extra process (chain profile only)one AKV-capable holderintactconsumers stay proto-only
B. Embed in every consumer−1 process, but the indexer needs a new home and protocolkey copied to N processesall in-process fakes destroyedSolana stack linked everywhere, incl. RCPT-only MXes
C1. Writes into sithbitd only2 chain-access paths foreverkey copied to every workerwrite-path fakes brokensithbitd links the full stack
C2. Optional in-process hosting in sithbitdboth hostings maintained foreverunchangedintactsithbitd links the full stack

Decision and rationale

Option A: the gateway stays a separate, private-network service.

  • The indexer forces a standing service anyway. The marginal cost of the read/write gateway roles riding in the same process is near zero; every integration variant still runs a process and pays migration.
  • Custody stays singular. One key, one process, Key-Vault-capable. Integration multiplies copies across workers and pushes the key-loading machinery into async consumer code for no custody gain.
  • The latency argument is empty. The hop is one loopback/private gRPC round-trip in front of a Solana RPC round-trip that is orders of magnitude larger and dominates every call.
  • The failure modes integration would “fix” are already correct. Gateway trouble at RCPT tempfails 451 4.4.3 (the sending MTA queues; operator trouble never bounces mail); sealing fails closed rather than falling back to plaintext; chain jobs retry from durable queues; every consumer dials lazily, so a gateway restart never crash-loops anything.
  • The proto seam is one of the workspace’s best assets — the in-process fakes keep four consumers’ test suites hermetic. Every non-A option damages or bifurcates it.

What would reopen this decision

  • The gateway ever needs to be reachable across a hostile network segment (no private network available): unauthenticated remote signing becomes untenable — add authentication, or revisit C1.
  • A deliberate product decision to ship a single all-in-one binary for operator simplicity: revisit C2.
  • Per-user signing keys replacing the process-wide operator key: the write gateway’s signing model dies regardless; redesign then.

Hardening queued with this decision

Three in-place improvements were queued rather than bundled here: (1) move mail-grpc onto the same TOML/env config layering as every other service, with dev-friendly in-code defaults (loopback bind, port 50051) so the private-only posture is the default rather than a convention — landed 2026-07-15 as a clean break (the legacy env-only configuration is gone; see the mail-grpc chapter); (2) make domain-sithbit’s delegate key loadable from a key source (file or Azure Key Vault) like the workspace’s other secrets — landed 2026-07-15 (see key sources); (3) give mail-grpc a presence in the infrastructure-as-code templates, in a private subnet — landed 2026-07-15 as an opt-in, BYO-network unit in both templates (ECS Fargate on AWS, a VNet-integrated ACI container group on Azure; see Provisioning with IaC). The related seam — chain-enabled sithbitd requiring an [ipfs] section even when an MX only wants RCPT verification — also landed 2026-07-15 with the config work: [grpc] alone is now the verification-only MX posture (see the chain-pipeline section).