Feed Format Draft Specification
Status: Draft v0.1 — subject to change Last updated: 2026-04-03
This document defines the message format for the proximity-based decentralized social application. It covers the envelope structure, encryption scheme, content types, wire format for BLE sync, and on-device storage schema.
1. Design Goals
The feed format must satisfy the following properties:
Tamper-evident. Messages form a cryptographic hash chain. Any modification to a historical message invalidates all subsequent message hashes, making tampering immediately detectable by any recipient who holds the chain.
Offline-composable. A user can author and sign messages without any network connection. Sequence numbers are local and monotonically increasing; no coordination with peers is required to produce a valid message.
Selectively readable. The same feed carries messages intended for different audiences — a 1-to-1 conversation, a group, all contacts, or only the author. Encryption is per-message with audience-specific key material. A peer can only decrypt messages addressed to an audience they belong to.
Efficient to sync. BLE has practical throughput of well under 1 Mbps once you account for connection overhead, MTU fragmentation, and retransmissions. Envelopes must be compact. Delta sync must be expressible in a small number of bytes. Fragmentation is defined at the wire layer rather than reinventing it per application.
Post-quantum safe. Classical key agreement (X25519) is accompanied by ML-KEM-768 in a hybrid scheme. If a quantum adversary breaks X25519 in the future, messages encrypted today remain confidential provided the ML-KEM component is sound.
Extensible. New message types can be introduced without breaking existing clients. Unknown types are stored and forwarded. The version field in the envelope allows future breaking changes to be versioned cleanly.
Compact. All binary fields use base64url (no padding) in the JSON representation, or raw bytes in the binary wire format. No redundant whitespace. Timestamps are Unix epoch integers, not ISO strings.
2. Identity and Keys
Every user has a single persistent identity and a set of keypairs derived at account creation time. All private key material is generated on-device and never leaves the device in plaintext.
2.1 Key Set
User identity:
identity_key: Ed25519 keypair
- Used for signing every message envelope
- Private key never leaves the secure enclave
- Public key is distributed to contacts and embedded in profiles
dh_key: X25519 keypair
- Used for Double Ratchet key agreement (1-to-1 messages)
- Rotated per-session by the ratchet; the device key is the "signed prekey"
- A fresh one-time prekey (X25519) is also generated per session initiation
pq_key: ML-KEM-768 keypair
- Post-quantum key encapsulation
- Used in hybrid KEM alongside X25519 for session establishment
- Combined shared secret: SHA-256(x25519_ss || ml_kem_ss)
feed_id: SHA-256(identity_key.public_bytes)
- Encoded as base64url (no padding), 43 characters
- Stable identifier for the feed; does not change with key rotation
- Analogous to SSB's @feed_id
2.2 Key Derivation and Storage
Key hierarchy:
Device seed (256-bit CSPRNG, stored in secure enclave)
|
+-- identity_key = Ed25519 key derived via HKDF(seed, "identity_key")
|
+-- dh_key = X25519 key derived via HKDF(seed, "dh_key")
|
+-- pq_key = ML-KEM-768 key derived via HKDF(seed, "pq_key")
|
+-- journal_key = 32-byte symmetric key via HKDF(seed, "journal_key")
(used for audience="self" encryption)
Private keys are stored in the platform secure enclave (iOS Secure Enclave, Android Keystore). Signing and decryption operations are performed inside the enclave; the raw private key bytes are not accessible to application code.
Public keys are distributed as part of the initial contact exchange (NFC/QR handshake) and are stored in the contacts table in the local SQLite database.
2.3 Key References in Messages
Messages reference the author by feed_id only. The public keys themselves are not embedded in individual messages — recipients who hold the contact record already have the public key, and new recipients cannot decrypt the message anyway.
The key_rotation message type (section 5) is the mechanism for announcing new public keys after a device migration.
3. Message Envelope
Every message, regardless of type or audience, is wrapped in a common envelope. The envelope fields are what get signed and chained. The content_enc field carries the encrypted payload.
3.1 Canonical JSON Structure
{
"version": 1,
"feed_id": "Zq3f8kR2mNpLvXwY1cBdHtAeOsUiGjKlP0FyVnWqCm4",
"sequence": 42,
"timestamp": 1743678000,
"previous": "7rT9xHmQ2sNpKvYw3cAdLtEeOuFiGjBlP1ZyVnWqCk5",
"type": "post",
"audience": "contacts",
"content_enc": "<base64url-encoded encrypted content>",
"signature": "<base64url-encoded Ed25519 signature>"
}
3.2 Field Definitions
version (integer, required)
Format version. Currently 1. Clients must reject messages with an unknown version. When a breaking change is made to the envelope structure, this field is incremented and a migration path is defined.
feed_id (string, required)
base64url(SHA-256(author identity_key.public_bytes)). Identifies whose feed this message belongs to. A message with a feed_id that does not match the signing key's derived feed_id is invalid and must be rejected.
sequence (integer, required)
Monotonically increasing, starts at 0 for the genesis message. There must be no gaps. A client receiving sequence N+2 when it holds N knows it is missing a message and must request the gap before processing.
timestamp (integer, required)
Unix epoch seconds. Note: offline devices may have skewed clocks. Timestamps are informational for display purposes only. Do not use timestamp comparisons for security decisions (e.g., freshness checks or ordering). Canonical ordering is by (feed_id, sequence).
previous (string or null, required)
null for the genesis message (sequence 0). For all other messages: base64url(SHA-256(canonical_serialization(previous_envelope))). This links the chain. A client must verify that SHA-256(envelope[N-1]) == envelope[N].previous before accepting message N. A broken chain is evidence of tampering or data loss.
type (string, required)
The message type, unencrypted. This is a routing hint only — it does not reveal message content. It allows clients to make decisions about storage priority and display without decrypting. Value is one of the types defined in section 5, or an unknown string (which clients must store but not render). Maximum 64 ASCII characters.
audience (string, required)
Describes who can decrypt content_enc. One of:
"self"— encrypted with journal_key, never replicated"direct:{peer_feed_id}"— Double Ratchet, visible only to that peer"group:{mls_group_id}"— MLS group, visible only to group members"contacts"— contacts symmetric key, visible to all direct contacts
content_enc (string, required)
base64url-encoded encrypted content bytes. The encryption scheme is determined by audience. See section 4.
signature (string, required)
base64url(Ed25519Sign(identity_key.private, canonical_serialization(envelope_without_signature))). The signature covers all other fields in their canonical serialization. See section 3.3.
3.3 Canonical Serialization for Signing
The signature is computed over a deterministic byte string derived from the envelope. This is necessary because JSON serialization is not canonical (key ordering and whitespace vary by implementation).
Canonical serialization rules:
- Collect all fields except
"signature". - Sort field names lexicographically (byte order, ASCII).
- Serialize as JSON with:
- No whitespace (no spaces, no newlines)
- Keys and string values in double quotes
- No trailing commas
- Integer values as bare numbers
nullasnull
- Encode the resulting UTF-8 string to bytes.
- Sign the bytes with Ed25519.
Sorted field order for the current envelope:
audience, content_enc, feed_id, previous, sequence, timestamp, type, version
Example canonical form (before signing):
{"audience":"contacts","content_enc":"<base64url>","feed_id":"<base64url>","previous":"<base64url>","sequence":42,"timestamp":1743678000,"type":"post","version":1}
Implementations must produce byte-identical serializations. Test vectors will be published separately.
3.4 Genesis Message
The first message in a feed (sequence 0) has previous: null. Its message ID anchors the entire chain. The genesis message is commonly a profile_update type, but this is not required.
{
"version": 1,
"feed_id": "Zq3f8kR2mNpLvXwY1cBdHtAeOsUiGjKlP0FyVnWqCm4",
"sequence": 0,
"timestamp": 1743678000,
"previous": null,
"type": "profile_update",
"audience": "contacts",
"content_enc": "<encrypted profile_update content>",
"signature": "<Ed25519 signature>"
}
3.5 Chain Integrity Diagram
Genesis Message 1 Message 2
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ seq: 0 │ │ seq: 1 │ │ seq: 2 │
│ previous: │ │ previous: │ │ previous: │
│ null │ H(M0) │ H(M0) ───┼──────┐ │ H(M1) ───┼──────┐
│ │◄─────────┼───────── │ │ │ │ │
│ sig: S(M0) │ │ sig: S(M1) │ │ │ sig: S(M2) │ │
└─────────────┘ └─────────────┘ │ └─────────────┘ │
│ │ │ │
└── H(M0) must equal M1.previous ───────┘ └── H(M1) ────┘
H = SHA-256(canonical_serialization(envelope))
S = Ed25519Sign(identity_key, canonical_serialization(envelope_without_sig))
4. Content Encryption
content_enc is always encrypted. The encryption scheme depends on the audience field. The plaintext passed to the encryption function is the canonical JSON serialization of the content object (section 5), encoded as UTF-8 bytes.
All AEAD operations use ChaCha20-Poly1305 (RFC 8439) unless otherwise specified. The nonce is always 96 bits derived from the message sequence number and feed_id to prevent nonce reuse.
4.1 Direct Messages (audience = "direct:{peer_feed_id}")
Direct messages use the Signal Protocol Double Ratchet Algorithm.
Session establishment:
- Alice retrieves Bob's key bundle from her contacts record:
- Bob's identity key (Ed25519, converted to X25519 for DH)
- Bob's signed prekey (X25519)
- Bob's one-time prekey (X25519, if available)
- Bob's PQ prekey (ML-KEM-768 public key)
- Alice performs X3DH key agreement:
DH1 = X25519(alice_identity_as_dh, bob_signed_prekey) DH2 = X25519(alice_ephemeral, bob_identity_as_dh) DH3 = X25519(alice_ephemeral, bob_signed_prekey) DH4 = X25519(alice_ephemeral, bob_one_time_prekey) [if available] - Alice performs ML-KEM-768 encapsulation:
(pq_ciphertext, pq_ss) = ML-KEM-768.Encapsulate(bob_pq_key) - Combined shared secret:
master_secret = HKDF( ikm = DH1 || DH2 || DH3 || DH4 || pq_ss, salt = "ProximityApp_X3DH_v1", info = alice_feed_id || bob_feed_id ) - Initialize Double Ratchet with
master_secret.
Per-message encryption:
message_key = DoubleRatchet.NextMessageKey()
nonce = derive_nonce(message.sequence, message.feed_id)
content_enc = ChaCha20Poly1305.Encrypt(message_key, nonce, content_bytes)
Wire format of content_enc for direct messages:
[ ratchet_header (variable) | encrypted_content ]
Ratchet header:
dh_public_key: 32 bytes (current ratchet DH public key)
message_number: 4 bytes (uint32, within current ratchet step)
prev_chain_length: 4 bytes (uint32, length of previous sending chain)
pq_ciphertext: 1088 bytes (ML-KEM-768 ciphertext, only on ratchet step)
[pq_present flag]: 1 byte (0x01 if pq_ciphertext present, 0x00 otherwise)
The entire wire format is then base64url encoded before being stored in content_enc.
4.2 Group Messages (audience = "group:{mls_group_id}")
Group messages use MLS (RFC 9420). The MLS group manages key material for all members.
Group state:
Each MLS group has an epoch. The epoch advances when members are added, removed, or when a member performs an update. Each epoch has a symmetric key derived by the MLS key schedule.
Encryption:
epoch_key = MLS.CurrentEpochKey(group_id)
sender_data_secret = MLS.SenderDataSecret(group_id)
content_enc = MLS_PrivateMessage(
group_id = group_id,
epoch = current_epoch,
content = content_bytes,
epoch_key = epoch_key
)
The content_enc value is a base64url-encoded MLSMessage of type PrivateMessage as defined in RFC 9420 Section 6.
Group management operations (add, remove, update) are carried in group_event messages (section 5) using MLSMessage of type PublicMessage or Commit.
Group identity:
The mls_group_id is a 32-byte random value chosen at group creation, base64url encoded. It is not the MLS group_id internal field directly but maps to it in the local keystore.
4.3 Contacts-Visible Messages (audience = "contacts")
Contacts messages are visible to all direct contacts of the author. This is intended for social feed posts shared broadly — analogous to a public timeline, but restricted to the contact graph.
Key establishment:
At contact-add time, both parties derive a shared contacts_key:
contacts_key = HKDF(
ikm = X25519(my_dh_key, their_dh_key),
salt = "ProximityApp_ContactsKey_v1",
info = sorted(my_feed_id, their_feed_id) // lexicographic sort
)
Each contact has a separate contacts_key. When the author sends an audience = "contacts" message, they encrypt the content once per contact using that contact's contacts_key. This means content_enc for contacts messages is actually a map:
{
"recipients": {
"<feed_id_1>": "<base64url encrypted content for contact 1>",
"<feed_id_2>": "<base64url encrypted content for contact 2>"
}
}
This map is itself base64url encoded and stored in content_enc. This is similar to SSB box2's recipient list model.
Key rotation: The contacts_key should be rotated every 30 days or after a contact is removed, by performing a new X25519 exchange using a fresh DH key and publishing a key_rotation message.
4.4 Self-Only Messages (audience = "self")
Self messages are never replicated. They are encrypted with a key derived from the user's seed:
journal_key = HKDF(device_seed, salt="ProximityApp_Journal_v1", length=32)
nonce = SHA-256(sequence || feed_id)[0:12] // first 12 bytes
content_enc = ChaCha20Poly1305.Encrypt(journal_key, nonce, content_bytes)
Because these are never sent to peers, the audience = "self" value in the envelope acts as a marker telling the sync layer to skip this message during replication.
5. Content Types
After decryption, content_enc yields a UTF-8 JSON object with a "type" field. The type in the content must match the type field in the outer envelope. This redundancy allows clients to verify the routing hint was not spoofed.
All content objects must have a "type" field as the first key. Unknown types must be stored and forwarded but not rendered. Additional unknown fields within a known type must be ignored (permissive parsing).
5.1 post — Social Feed Post
{
"type": "post",
"body": "Just met someone interesting at the conference. Physical-first social really works.",
"attachments": [
"uK9mP2xQnRsLtYvWzCdHeAfObNgJiFlD3EpVrXwGjBq0"
],
"reply_to": "7rT9xHmQ2sNpKvYw3cAdLtEeOuFiGjBlP1ZyVnWqCk5",
"mentions": [
"Zq3f8kR2mNpLvXwY1cBdHtAeOsUiGjKlP0FyVnWqCm4"
]
}
| Field | Type | Constraints |
|---|---|---|
body |
string | Required. Max 2000 UTF-8 characters. |
attachments |
array of strings | Optional. Each entry is base64url(SHA-256(blob)). Max 4 attachments. Blobs stored separately (section 9). |
reply_to |
string or null | Optional. message_id of the message being replied to. |
mentions |
array of strings | Optional. feed_id values of mentioned users. No semantic effect on encryption. |
5.2 reaction — Emoji Reaction
{
"type": "reaction",
"target_message": "7rT9xHmQ2sNpKvYw3cAdLtEeOuFiGjBlP1ZyVnWqCk5",
"emoji": "👍"
}
| Field | Type | Constraints |
|---|---|---|
target_message |
string | Required. message_id of the target message. |
emoji |
string | Required. Single Unicode emoji character (1–4 codepoints). |
5.3 profile_update — Display Name or Avatar
{
"type": "profile_update",
"display_name": "Alice",
"avatar_hash": "uK9mP2xQnRsLtYvWzCdHeAfObNgJiFlD3EpVrXwGjBq0"
}
| Field | Type | Constraints |
|---|---|---|
display_name |
string or null | Optional. Max 64 UTF-8 characters. Shown only to contacts who can decrypt this message. |
avatar_hash |
string or null | Optional. base64url(SHA-256(avatar image bytes)). Image fetched from blob store. |
The most recent profile_update in a feed supersedes all previous ones. Clients should index this for fast lookup.
5.4 contact_introduction — Introduce Two Contacts
{
"type": "contact_introduction",
"introduced_feed_id": "pL4rN7xJmQsKuYwZvCdHfAbOgNgIiFkD3EpVrXwGjBq8",
"introduced_dh_key": "xKpR2mNqLsYvXwZtCdHeAfOuGjBlP0FyVnWqCk5Tg9",
"introduced_pq_key": "...ML-KEM-768 public key bytes, base64url...",
"note": "We met at OpenTech Berlin. Highly recommend connecting."
}
| Field | Type | Constraints |
|---|---|---|
introduced_feed_id |
string | Required. feed_id of the person being introduced. |
introduced_dh_key |
string | Required. Their X25519 public key, base64url. |
introduced_pq_key |
string | Required. Their ML-KEM-768 public key, base64url. |
note |
string or null | Optional. Max 256 UTF-8 characters. Context for the introduction. |
The recipient of this message can use the provided keys to initiate an X3DH session with the introduced party during a subsequent in-person encounter. The introduction does not bypass the physical-first requirement — it merely pre-stages key material.
5.5 key_rotation — Announce New Keypair
{
"type": "key_rotation",
"new_identity_key": "...new Ed25519 public key, base64url...",
"new_dh_key": "...new X25519 public key, base64url...",
"new_pq_key": "...new ML-KEM-768 public key, base64url...",
"previous_key_sig": "...Ed25519 signature by old identity key over new keys, base64url...",
"reason": "device_migration"
}
| Field | Type | Constraints |
|---|---|---|
new_identity_key |
string | Required. New Ed25519 public key, base64url. |
new_dh_key |
string | Required. New X25519 public key, base64url. |
new_pq_key |
string | Required. New ML-KEM-768 public key, base64url. |
previous_key_sig |
string or null | Optional but strongly recommended. Signature by the old identity key over the concatenation of the three new public keys. null if the old device was lost and the old key is unavailable. |
reason |
string | Optional. One of "device_migration", "key_compromise", "scheduled_rotation". |
Verification: Contacts who receive a key_rotation message must verify previous_key_sig using the old identity key if it is available. A rotation without a valid previous_key_sig must be flagged as unverified (but not automatically rejected — device loss is a real scenario).
The feed_id does not change after key rotation. It continues to be SHA-256 of the original identity key, serving as a stable long-term identifier.
5.6 group_event — MLS Group Management
{
"type": "group_event",
"group_id": "hF3kP8mRnLsYvXwZtCdHeAfOuGjIlD2EpVqBqCk5Tg9",
"event": "add_member",
"mls_message": "...base64url encoded MLSMessage..."
}
| Field | Type | Constraints |
|---|---|---|
group_id |
string | Required. 32-byte random group identifier, base64url. |
event |
string | Required. One of "create", "add_member", "remove_member", "update_key". |
mls_message |
string | Required. base64url-encoded MLS MLSMessage (RFC 9420). |
Group events must be processed in order. Each member's MLS state machine processes the MLSMessage to advance the epoch. The event field is a routing hint matching the MLS message type.
5.7 tombstone — Soft-Delete a Message
{
"type": "tombstone",
"target_message": "7rT9xHmQ2sNpKvYw3cAdLtEeOuFiGjBlP1ZyVnWqCk5",
"reason": "retracted"
}
| Field | Type | Constraints |
|---|---|---|
target_message |
string | Required. message_id of the message to be soft-deleted. |
reason |
string | Optional. One of "retracted", "error", "spam". |
Important limitations: A tombstone signals the author's intent to retract a message. It cannot remove the message from peers' devices that have already stored it. Clients should:
- Hide the original message from the feed display.
- Retain the original message in storage (for chain integrity and audit).
- Mark the original message as tombstoned in the database.
Tombstones only work forward in time — recipients who have not yet synced the original message will never receive the tombstoned content if their client respects the tombstone ordering.
6. Message ID Scheme
Message IDs are deterministic cryptographic hashes of the full canonical envelope.
Derivation:
message_id = base64url(SHA-256(canonical_serialization(envelope)))
Where canonical_serialization applies the same rules as in section 3.3, but includes all fields including "signature".
Properties:
- Globally unique (probabilistic): SHA-256 has 256-bit preimage resistance. Collision probability across the entire network is negligible.
- Deterministic: Given the same signed envelope bytes, every client derives the same message ID.
- Self-authenticating: Any client can verify a message ID by recomputing SHA-256 of the envelope it holds.
- Used in:
reply_to,target_messagein reactions and tombstones,previouschain link, sync protocol references.
Encoding: base64url without padding (43 ASCII characters for SHA-256 output).
7. Comparison with Secure Scuttlebutt (SSB)
This format is architecturally inspired by SSB but diverges significantly in its privacy and cryptography model.
| Property | SSB | This Format |
|---|---|---|
| Append-only log | Yes | Yes |
| Hash chain | Yes (SHA-256) | Yes (SHA-256) |
| Ed25519 signing | Yes | Yes |
| Feed content | Plaintext JSON | Encrypted (AEAD per audience) |
| Replication scope | Friends-of-friends gossip | Direct contacts only (no transitive relay of content) |
| Private messages | Bolt-on (box2, post-hoc) | Native (Double Ratchet for DM, MLS for groups) |
| Group messages | box2 (limited, no forward secrecy) | MLS RFC 9420 (full key schedule, PCS, FS) |
| Post-quantum | No | Yes (hybrid X25519 + ML-KEM-768) |
| Message deletion | No (append-only, no tombstone) | Tombstone (soft delete, intent signaled) |
| Feed IDs | Permanent @<pubkey>.ed25519 |
Permanent SHA-256(pubkey), survives key rotation |
| Transport layer | TCP, WebSocket, pub servers | BLE, WiFi Direct, optionally Tor |
| Discovery | Pubs, DHT proposals | Physical proximity only (NFC/QR/BLE) |
| Metadata exposure | Message graph visible to all replicators | Message graph visible only to direct contacts |
| Canonical encoding | JSSB (custom) | Sorted-key JSON, UTF-8 |
| Forward secrecy | No | Yes (Double Ratchet for DM; MLS epoch ratchet for groups) |
The most significant departure is that content is never available in plaintext to relaying peers. In SSB, any pub server or friend-of-friend replicating your feed can read all of your posts. In this format, a relaying peer (e.g., a contact helping forward your messages to a mutual) receives only ciphertext they cannot decrypt.
8. Wire Format for BLE Sync
BLE 5.x has a theoretical PHY rate of 2 Mbps, but practical application-layer throughput after connection overhead, ATT MTU negotiation, and retransmissions is typically 20–200 kbps. Messages must be fragmented for transport.
8.1 MTU and Fragmentation
The ATT MTU is negotiated at connection time, typically 23–512 bytes (with 3 bytes for ATT overhead, leaving 20–509 bytes of payload per packet). We target a conservative minimum of 20 bytes payload per chunk but design for 244 bytes (BLE 5 Data Length Extension default).
8.2 Sync Handshake
Alice Bob
| |
|---- SyncRequest(feed_id, N) >| "I have seq 0..N for this feed"
| |
|< --- SyncOffer(feed_id, M) --| "I have seq 0..M, will send N+1..M"
| |
|< --- SyncChunk (chunk 1/K) --|
|< --- SyncChunk (chunk 2/K) --|
| ... |
|< --- SyncChunk (chunk K/K) --|
| |
|---- SyncAck(message_id) ---> | "Received and verified message"
| ... |
Multiple feeds can be interleaved in a single BLE session. More recent messages are sent first within a feed (descending sequence order), allowing a peer with limited time to receive the freshest content.
8.3 Binary Wire Structures
All multi-byte integers are big-endian.
SyncRequest (36 bytes):
┌─────────────────────────────────────────────┐
│ feed_id 32 bytes (raw SHA-256 bytes) │
│ have_seq 4 bytes (uint32) │
└─────────────────────────────────────────────┘
have_seq is the highest sequence number the sender holds for this feed. Use 0xFFFFFFFF to indicate "I have nothing" (request entire feed). Use the actual last sequence number to request only delta.
SyncOffer (40 bytes):
┌──────────────────────────────────────────────────┐
│ feed_id 32 bytes (raw SHA-256 bytes) │
│ have_seq 4 bytes (uint32, sender's latest)│
│ message_count 4 bytes (uint32, how many follow) │
└──────────────────────────────────────────────────┘
SyncChunk (variable, min 37 bytes):
┌────────────────────────────────────────────────────────────────────┐
│ message_id 32 bytes (raw SHA-256 bytes of envelope) │
│ chunk_index 2 bytes (uint16, 0-indexed fragment number) │
│ total_chunks 2 bytes (uint16, total fragments for this msg) │
│ data N bytes (up to MTU - 36 bytes of envelope bytes) │
└────────────────────────────────────────────────────────────────────┘
For a 244-byte MTU payload: data is up to 208 bytes per chunk, giving a maximum message size of 208 * 65535 ≈ 13 MB. In practice, messages should be under 64 KB. Attachments (blobs) are transferred separately.
SyncAck (33 bytes):
┌──────────────────────────────────────────────────┐
│ message_id 32 bytes (raw SHA-256 bytes) │
│ status 1 byte (0x00=ok, 0x01=invalid) │
└──────────────────────────────────────────────────┘
status = 0x01 indicates the received message failed signature or chain verification. The sender should log the rejection.
8.4 Compression
Encrypted content may appear random and may not compress well. However, the JSON envelope structure (field names, base64url encoding) does compress. Implementations may apply zstd (level 1 for speed) to the entire envelope before fragmentation. A 1-byte flag in a future wire format version will signal compression presence.
Do not compress blobs (images and other attachments are typically already compressed).
8.5 Priority and Scheduling
Within a sync session, send messages in this priority order:
- Messages from the remote peer's own feed (they sent us their feed, we send ours)
- Messages of type
key_rotation,group_event(key material, high priority) - Messages of type
post,reaction(content, in descending sequence order) - Messages of type
tombstone,profile_update(low urgency) - Blobs (lowest priority, skip if session is short)
9. Storage Schema
On-device storage uses SQLite. SQLite is appropriate for mobile (ACID, single-writer, excellent library support on iOS and Android), and the data volumes involved (tens of thousands of messages) are well within its sweet spot.
9.1 Schema
-- Core message table
CREATE TABLE messages (
message_id TEXT PRIMARY KEY, -- base64url(SHA-256(envelope))
feed_id TEXT NOT NULL, -- base64url(SHA-256(identity_key.pub))
sequence INTEGER NOT NULL,
timestamp INTEGER NOT NULL, -- Unix epoch seconds
type TEXT NOT NULL, -- unencrypted type hint from envelope
audience TEXT NOT NULL, -- unencrypted audience from envelope
envelope_json TEXT NOT NULL, -- full signed envelope, JSON string
content_json TEXT, -- decrypted content JSON, NULL if not decryptable
tombstoned INTEGER DEFAULT 0, -- 1 if a tombstone exists for this message
received_at INTEGER NOT NULL, -- local Unix epoch millis when stored
UNIQUE(feed_id, sequence)
);
CREATE INDEX idx_messages_feed_seq ON messages(feed_id, sequence);
CREATE INDEX idx_messages_timestamp ON messages(timestamp);
CREATE INDEX idx_messages_type ON messages(type);
CREATE INDEX idx_messages_received_at ON messages(received_at);
-- Blob storage for attachments
CREATE TABLE blobs (
content_hash TEXT PRIMARY KEY, -- base64url(SHA-256(data))
data BLOB NOT NULL,
mime_type TEXT,
size_bytes INTEGER NOT NULL,
stored_at INTEGER NOT NULL -- local Unix epoch millis
);
-- Contact records
CREATE TABLE contacts (
feed_id TEXT PRIMARY KEY,
display_name TEXT, -- from most recent profile_update
avatar_hash TEXT, -- content_hash of avatar blob
identity_key_pub TEXT NOT NULL, -- Ed25519 public key, base64url
dh_key_pub TEXT NOT NULL, -- X25519 public key, base64url
pq_key_pub TEXT NOT NULL, -- ML-KEM-768 public key, base64url
contacts_key TEXT NOT NULL, -- shared symmetric key for audience=contacts
added_at INTEGER NOT NULL,
last_seen_seq INTEGER DEFAULT -1 -- highest sequence number synced from this feed
);
-- MLS group membership
CREATE TABLE mls_groups (
group_id TEXT PRIMARY KEY, -- base64url random ID
group_name TEXT,
epoch INTEGER NOT NULL DEFAULT 0,
mls_state_blob BLOB NOT NULL, -- serialized MLS group state (opaque)
created_at INTEGER NOT NULL
);
CREATE TABLE mls_group_members (
group_id TEXT NOT NULL REFERENCES mls_groups(group_id),
feed_id TEXT NOT NULL,
joined_at_epoch INTEGER NOT NULL,
PRIMARY KEY(group_id, feed_id)
);
-- Double Ratchet session state per contact
CREATE TABLE dr_sessions (
peer_feed_id TEXT PRIMARY KEY REFERENCES contacts(feed_id),
session_state BLOB NOT NULL, -- serialized Double Ratchet state (opaque)
last_updated INTEGER NOT NULL
);
9.2 Key Material Storage
Key material is never stored in SQLite. All private keys (identity_key, dh_key, pq_key, journal_key) are stored in:
- iOS: Secure Enclave backed by device hardware (T2/M-series chip). Keys are created with
kSecAttrAccessibleWhenUnlockedThisDeviceOnlyandkSecAttrTokenIDSecureEnclave. - Android: Android Keystore System. Private key operations happen inside the Keystore and key bytes are not exposed to application code.
The contacts_key (shared symmetric key) is a 32-byte symmetric value. It may be stored encrypted with a key derived from the identity key, in the platform keychain (iOS Keychain, Android EncryptedSharedPreferences) rather than in SQLite.
9.3 Storage Limits and Garbage Collection
- Maximum stored messages per contact feed: 10,000 (configurable)
- Maximum blob store size: 500 MB (configurable)
- Tombstoned messages: retain the envelope row (for chain integrity) but delete
content_json - Blobs not referenced by any message: eligible for GC after 7 days
10. Versioning and Forward Compatibility
10.1 Envelope Version
The version field in the envelope is a positive integer. The current version is 1.
Version policy:
- Additive changes (new optional fields, new
typevalues, newaudienceprefixes): no version bump required. Old clients ignore unknown fields. - Breaking changes (changed field semantics, removed required fields, changed signing scheme): require a version bump. Clients must reject envelopes with an unknown version.
- A client that encounters
version > 1must store the envelope as-is (for relay) but must not attempt to verify the signature or decrypt the content using v1 logic.
10.2 Unknown Message Types
When a client receives a message with an unknown type value:
- Verify the chain (check
previoushash andsignature). - Store the full
envelope_jsonin the messages table. - Do not attempt to decrypt or render the message.
- Do not surface the message in the UI.
- Include the message when relaying to contacts (the recipient may have a newer client that understands the type).
This allows new message types to be rolled out gradually without breaking existing clients' chain integrity or relay behavior.
10.3 Unknown Content Fields
Within a known message type, clients must apply permissive parsing:
- Unknown JSON keys in the content object are silently ignored.
- Missing optional fields are treated as
null. - A
typefield in content that does not match the envelopetypeis a hard error (reject and do not store).
10.4 Migration Path for Version Bumps
When version 2 is introduced:
- A
version_migrationmessage type (in v1) can announce to existing contacts that the feed will begin emitting v2 messages. - Clients that understand v2 process the new messages normally.
- v1-only clients store and relay v2 envelopes without processing them, preserving chain integrity and replication for the network.
10.5 Deprecation
Fields and types are deprecated by:
- Marking them deprecated in this specification with a version annotation.
- Continuing to support them for at least two major version cycles.
- Eventually treating deprecated types as unknown (store and relay, do not render).
Appendix A: Cryptographic Primitive Summary
| Purpose | Algorithm | Parameters |
|---|---|---|
| Identity signing | Ed25519 | RFC 8032 |
| Classical DH | X25519 | RFC 7748 |
| Post-quantum KEM | ML-KEM-768 | NIST FIPS 203 |
| Hybrid shared secret | HKDF-SHA-256 | RFC 5869 |
| AEAD encryption | ChaCha20-Poly1305 | RFC 8439 |
| Hash chain / IDs | SHA-256 | FIPS 180-4 |
| Key derivation | HKDF-SHA-256 | RFC 5869 |
| 1-to-1 messaging | Double Ratchet + X3DH | Signal Protocol spec |
| Group messaging | MLS | RFC 9420 |
Appendix B: Encoding Reference
All binary values are encoded as base64url (RFC 4648 Section 5) without padding when embedded in JSON. In binary wire format, raw bytes are used directly.
| Value | JSON encoding | Wire encoding | Size |
|---|---|---|---|
| SHA-256 hash | base64url, 43 chars | 32 bytes raw | 32 bytes |
| Ed25519 public key | base64url, 43 chars | 32 bytes raw | 32 bytes |
| Ed25519 signature | base64url, 86 chars | 64 bytes raw | 64 bytes |
| X25519 public key | base64url, 43 chars | 32 bytes raw | 32 bytes |
| ML-KEM-768 public key | base64url, ~1568 chars | 1184 bytes raw | 1184 bytes |
| ML-KEM-768 ciphertext | base64url, ~1452 chars | 1088 bytes raw | 1088 bytes |
Appendix C: Example Full Message
A complete signed post message (all binary values abbreviated):
{
"version": 1,
"feed_id": "Zq3f8kR2mNpLvXwY1cBdHtAeOsUiGjKlP0FyVnWqCm4",
"sequence": 7,
"timestamp": 1743678000,
"previous": "7rT9xHmQ2sNpKvYw3cAdLtEeOuFiGjBlP1ZyVnWqCk5",
"type": "post",
"audience": "contacts",
"content_enc": "eyJyZWNpcGllbnRzIjp7IlpxM2Y4a1IybU5w...<truncated>",
"signature": "Ln4kP9mRsYvXwZtCdHeAfOuGjBlD2EpVqBqCk5Tg9...<truncated>"
}
Decrypted content (after decryption by an authorized contact):
{
"type": "post",
"body": "Just shipped the BLE sync handshake. Range is about 8m through walls.",
"attachments": [],
"reply_to": null,
"mentions": []
}