1. .md files remain the sole source of truth — SQLite is never synced, always rebuilt locally
2. Local-first — the app works fully offline; sync is additive, never blocking
3. We control the sync — custom API + S3 storage, no dependency on platform cloud behavior
4. Conflicts are visible, not silent — users must see and resolve merge conflicts
5. Client does the heavy lifting — the API is a thin coordination layer; file bytes go directly to/from S3 via presigned URLs
6. Cross-platform from day one — nothing Apple-specific in the sync protocol
```
┌──────────────────────────────────────────────────────────────────┐
│ Client (per device) │
│ │
│ VaultManager / SearchEngine / VaultIndexer │
│ │ │
│ ┌─────▼──────┐ │
│ │ SyncManager │ ← orchestrator │
│ └─────┬──────┘ │
│ │ │
│ ┌──────┼──────────┐ │
│ │ │ │ │
│ ┌──▼──┐ ┌─▼────┐ ┌──▼───┐ │
│ │Chang│ │Confli│ │Index │ │
│ │Detec│ │Resol │ │Rebui │ │
│ └──┬──┘ └──┬───┘ └──┬───┘ │
│ └────────┼────────┘ │
│ │ │
│ ┌──────▼──────┐ │
│ │ SyncClient │ ← Ktor HTTP client │
│ └──────┬──────┘ │
└─────────────┼────────────────────────────────────────────────────┘
│
│ HTTPS
│
┌─────────────▼────────────────────────────────────────────────────┐
│ Backend │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ Sync API │────▶│ Postgres │ │ S3 / R2 │ │
│ │ (REST) │ │ (users, │ │ (vault files) │ │
│ │ │──┐ │ devices, │ │ │ │
│ └──────────────┘ │ │ file state)│ │ /{user_id}/ │ │
│ │ └──────────────┘ │ sources/ │ │
│ │ │ thoughts/ │ │
│ │ ┌──────────────┐ │ questions/ │ │
│ └─▶│ APNs / FCM │ │ syntheses/ │ │
│ │ (push) │ └──────────────────┘ │
│ └──────────────┘ │
└──────────────────────────────────────────────────────────────────┘
```
**Full auth architecture has moved to AUTH_ARCHITECTURE.md.**
Summary: multi-provider auth supporting CLI (email/password), Sign in with Apple, and future OAuth providers. One user can link multiple sign-in methods. All providers issue the same JWT — downstream code only sees user_id.
Key change from the original design:userstable no longer hasprovider/provider_idcolumns. Those live in a separateauth_providerstable to support account linking.
The API is a thin coordination layer. It never reads or writes vault file contents — that goes directly to S3.
POST /auth/loginExchange identity provider token for access/refresh tokens.
POST /auth/refreshExchange refresh token for new access token.
POST /devices/registerRegister this device for push notifications.
```json
{
"device_name": "Ethan's iPhone",
"platform": "ios",
"push_token": "abc123..."
}
```
GET /vault/stateReturns the server's view of the user's vault — every file with its current version.
Response:
```json
{
"files": [
{
"path": "sources/beyond-good-and-evil.md",
"version": 3,
"content_hash": "sha256:abc123...",
"size_bytes": 2048,
"updated_at": "2026-03-24T10:30:00Z",
"updated_by_device": "Ethan's MacBook"
}
],
"deleted_since": [
{
"path": "thoughts/tht-2026-03-20-001.md",
"deleted_at": "2026-03-23T15:00:00Z",
"deleted_by_device": "Ethan's iPhone"
}
],
"server_time": "2026-03-24T12:00:00Z"
}
```
The client can optionally pass ?since={ISO timestamp} to get only changes since last sync (incremental). Without it, the full state is returned.
POST /vault/pushClient wants to upload one or more files. API validates, returns presigned S3 upload URLs.
Request:
```json
{
"files": [
{
"path": "sources/new-article.md",
"content_hash": "sha256:def456...",
"size_bytes": 1024,
"base_version": null
},
{
"path": "sources/beyond-good-and-evil.md",
"content_hash": "sha256:ghi789...",
"size_bytes": 2100,
"base_version": 3
}
]
}
```
Response:
```json
{
"uploads": [
{
"path": "sources/new-article.md",
"upload_url": "https://s3.../presigned-put-url",
"new_version": 1
},
{
"path": "sources/beyond-good-and-evil.md",
"upload_url": "https://s3.../presigned-put-url",
"new_version": 4
}
],
"conflicts": []
}
```
If base_version doesn't match the server's current version, the file is returned in conflicts instead of uploads:
```json
{
"uploads": [...],
"conflicts": [
{
"path": "sources/beyond-good-and-evil.md",
"your_base_version": 3,
"server_version": 4,
"server_hash": "sha256:xyz...",
"download_url": "https://s3.../presigned-get-url"
}
]
}
```
The client must resolve the conflict before it can push that file.
POST /vault/push/confirmAfter uploading file bytes to S3, the client confirms the push so the API updates version state and notifies other devices.
Request:
```json
{
"confirmed": [
{ "path": "sources/new-article.md", "version": 1 },
{ "path": "sources/beyond-good-and-evil.md", "version": 4 }
]
}
```
The API then:
1. Updates file_state in Postgres
2. Sends push notification to all other devices for this user
POST /vault/pullClient wants to download files. API returns presigned S3 download URLs.
Request:
```json
{
"files": [
"sources/new-article.md",
"thoughts/tht-2026-03-24-001.md"
]
}
```
Response:
```json
{
"downloads": [
{
"path": "sources/new-article.md",
"download_url": "https://s3.../presigned-get-url",
"version": 1,
"content_hash": "sha256:abc...",
"size_bytes": 1024
}
]
}
```
POST /vault/deleteClient deleted files locally, propagate to server.
Request:
```json
{
"files": [
{ "path": "thoughts/tht-2026-03-20-001.md", "base_version": 2 }
]
}
```
Same version check as push — if base_version doesn't match, it's a conflict (file was edited on another device after this device last synced).
GET /vault/exportReturns a presigned URL for a zip of the entire vault. For data portability.
```sql
CREATE TABLE file_state (
user_id UUID NOT NULL REFERENCES users(id),
path TEXT NOT NULL, -- relative path in vault
version INTEGER NOT NULL DEFAULT 1,
content_hash TEXT NOT NULL, -- SHA-256
size_bytes BIGINT NOT NULL,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_by UUID REFERENCES devices(id),
is_deleted BOOLEAN NOT NULL DEFAULT false,
deleted_at TIMESTAMPTZ,
PRIMARY KEY (user_id, path)
);
-- Tombstones are kept for 90 days so devices that haven't synced
-- in a while can still learn about deletions.
CREATE INDEX idx_file_state_deleted ON file_state(user_id, is_deleted, deleted_at);
```
```
steel-notes-vaults/
└── {user_id}/
├── sources/
│ ├── beyond-good-and-evil.md
│ └── thinking-fast-and-slow.md
├── thoughts/
│ └── tht-2026-03-20-001.md
├── questions/
│ └── qst-2026-03-20-001.md
└── syntheses/
└── syn-2026-03-20-001.md
```
Each file is stored at its natural vault path, prefixed by user ID. No versioning in S3 — Postgres file_state.version is the version authority. (S3 versioning can be enabled as a backup safety net but is not part of the sync protocol.)
When a push/confirm or delete is processed, the API sends a silent push to all other devices for that user:
APNs (iOS/macOS):
```json
{
"aps": { "content-available": 1 },
"steel-notes": {
"event": "vault-changed",
"changed_paths": ["sources/new-article.md"],
"server_time": "2026-03-24T12:00:00Z"
}
}
```
FCM (Android, future):
Same payload structure, delivered as a data message.
The client receives this in the background, triggers SyncManager.syncNow(), and pulls the changed files. From the user's perspective, notes appear on other devices within seconds.
Lives in commonMain. Thin wrapper around the API endpoints.
```
SyncClient
├── login(identityToken: String): AuthTokens
├── refreshToken(refreshToken: String): AuthTokens
├── registerDevice(name: String, platform: String, pushToken: String?)
├── getVaultState(since: Instant?): VaultState
├── requestPush(files: List<PushRequest>): PushResponse // returns presigned URLs
├── confirmPush(confirmed: List<FileVersion>)
├── requestPull(paths: List<String>): PullResponse // returns presigned URLs
├── deleteFiles(files: List<DeleteRequest>): DeleteResponse
├── uploadToS3(url: String, content: ByteArray) // direct S3 PUT
├── downloadFromS3(url: String): ByteArray // direct S3 GET
└── exportVault(): String // returns download URL
```
```
SyncManager
├── Properties
│ ├── syncClient: SyncClient
│ ├── conflictResolver: ConflictResolver
│ ├── syncState: SyncStateDatabase // .steel/sync.db
│ ├── status: StateFlow<SyncStatus> // exposed to UI
│ └── pendingConflicts: StateFlow<List<SyncConflict>>
│
├── Lifecycle
│ ├── start() → load sync state, trigger initial sync
│ ├── stop() → flush pending writes
│ └── syncNow() → full sync cycle (called on push notification or pull-to-refresh)
│
├── Full Sync Cycle (syncNow)
│ │
│ │ ── Phase 1: Detect local changes ──
│ ├── Scan local vault files → compute hashes
│ ├── Compare against sync_state table
│ ├── Build list of locally changed/added/deleted files
│ │
│ │ ── Phase 2: Get remote state ──
│ ├── GET /vault/state?since={last_sync_time}
│ ├── Compare remote state against sync_state
│ ├── Build list of remotely changed/added/deleted files
│ │
│ │ ── Phase 3: Resolve ──
│ ├── Files changed only locally → push
│ ├── Files changed only remotely → pull
│ ├── Files changed on both sides → conflict resolution
│ ├── Files deleted remotely, unchanged locally → delete local
│ ├── Files deleted remotely, changed locally → conflict
│ ├── Files deleted locally, unchanged remotely → push delete
│ ├── Files deleted locally, changed remotely → conflict
│ │
│ │ ── Phase 4: Execute ──
│ ├── POST /vault/push → get presigned URLs → upload to S3 → confirm
│ ├── POST /vault/pull → get presigned URLs → download from S3 → write locally
│ ├── POST /vault/delete → propagate deletions
│ ├── Update sync_state for all resolved files
│ ├── Trigger VaultIndexer for changed/added/deleted files
│ │
│ │ ── Phase 5: Record ──
│ └── Update last_sync_time
│
├── onLocalFileChanged(path)
│ ├── Update sync_state → pending_push
│ ├── Debounce 2s (user may be mid-edit)
│ └── Run sync cycle (push only for this file)
│
├── onPushNotificationReceived(changedPaths)
│ └── Run sync cycle (pull only for changed paths)
│
└── onConflictResolved(path, mergedContent)
├── Write merged file locally
├── Push to server (base_version = server's version since we've seen it)
├── Update sync_state → synced
└── Reindex note
```
```
enum SyncStatus {
IDLE, // All files synced
SYNCING, // Active push/pull in progress
OFFLINE, // No network, changes queued locally
ERROR, // API error (auth, quota, server down)
UNAUTHENTICATED,// Not logged in
CONFLICT // One or more files need manual resolution
}
```
.steel/sync.db)```sql
CREATE TABLE sync_state (
relative_path TEXT PRIMARY KEY,
content_hash TEXT NOT NULL, -- SHA-256 of local file
server_version INTEGER NOT NULL, -- version from server (0 if never pushed)
sync_status TEXT NOT NULL, -- 'synced' | 'pending_push' | 'pending_pull' | 'conflict'
last_synced_at TEXT -- ISO-8601, null if never synced
);
CREATE TABLE sync_meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL
);
-- Keys: 'last_sync_time', 'user_id', 'device_id'
CREATE TABLE sync_log (
id INTEGER PRIMARY KEY AUTOINCREMENT,
timestamp TEXT NOT NULL,
event_type TEXT NOT NULL, -- 'push' | 'pull' | 'conflict_resolved' | 'delete' | 'error'
relative_path TEXT NOT NULL,
detail TEXT
);
```
Unchanged from the original design — this is entirely client-side logic and doesn't depend on the transport backend.
A conflict occurs when POST /vault/push returns a file in conflicts[] because base_version doesn't match the server's version. This means another device pushed a change since this device last synced.
Since our .md files have a consistent structure (YAML frontmatter + markdown body), we can do smarter merging than generic text diff:
```
ConflictResolver
├── resolve(localContent: String, remoteContent: String, baseContent: String?): MergeResult
│ ├── Split each version into frontmatter + body
│ ├── Merge frontmatter fields independently:
│ │ ├── Non-conflicting field changes → auto-merge
│ │ ├── Same field changed to same value → auto-merge
│ │ └── Same field changed to different values → CONFLICT
│ ├── Merge body:
│ │ ├── Only one side changed body → take that side
│ │ ├── Both changed, non-overlapping regions → auto-merge (if base available)
│ │ └── Both changed, overlapping regions → CONFLICT
│ └── Return MergeResult
```
baseContent is available when the client kept the previous version before editing. The client should store this in sync.db or a shadow file for three-way merge support.
```
sealed interface MergeResult {
data class AutoMerged(val merged: String) : MergeResult
data class NeedsUserInput(
val localVersion: String,
val remoteVersion: String,
val baseVersion: String?,
val conflictDescription: String
) : MergeResult
}
```
| Scenario | Resolution |
|---|---|
Only frontmatter updated timestamp differs | Take the later timestamp |
| Tags added on one device, different tags added on other | Union of both tag sets |
| Status changed on one device, body edited on other | Accept both changes |
| Annotations added on one device, body edited on other | Accept both changes |
| Body edited in non-overlapping sections (3-way merge) | Merge both changes |
| Scenario | UI Treatment |
|---|---|
| Same paragraph edited differently on two devices | Side-by-side diff, user picks or edits |
| Status changed to different values on two devices | Show both options, user picks |
| Note deleted on one device, edited on other | "Deleted on [device]. Keep the edited version?" |
| Body rewritten substantially on both | Show both, user picks or combines |
1. Note appears in vault list with a conflict badge
2. Tapping opens ConflictResolutionView:
3. After resolution, merged file is pushed via normal sync
Conflicts never block the app. The local version is used while conflicts queue.
1. User opens app → sees onboarding → signs in with Apple
2. Client calls POST /auth/login → gets tokens + user_id
3. Client calls POST /devices/register → registers for push
4. GET /vault/state returns empty → client initializes local vault
5. User creates first note → triggers push sync
1. Signs in → POST /auth/login returns existing user_id
2. Registers device for push
3. GET /vault/state returns all files → client pulls entire vault via presigned URLs
4. Show progress: "Downloading 47 of 312 notes..."
5. After pull completes → VaultIndexer.fullReindex() → ready
1. User had a local vault before accounts existed (or was offline-only)
2. On sign-in, detect existing local vault with files
3. Prompt: "Upload your existing vault to sync across devices?"
4. Push all local files → server creates file_state entries
commonMain)```
com.steelnotes.sync/
├── SyncManager.kt // Orchestrator (full sync cycle, lifecycle)
├── SyncClient.kt // Ktor HTTP client for Sync API
├── SyncModels.kt // Data classes (SyncFile, SyncEvent, SyncStatus, API DTOs)
├── SyncStateDatabase.sq // SQLDelight schema for .steel/sync.db
├── ChangeDetector.kt // Hash-based local change detection
└── ConflictResolver.kt // Frontmatter-aware three-way merge
com.steelnotes.auth/
├── AuthManager.kt // Token storage, refresh, login flow
├── AuthModels.kt // AuthTokens, User, Device
└── TokenStore.kt // expect/actual — Keychain (Apple) / Keystore (Android)
```
appleMain)```
com.steelnotes.auth/
└── AppleTokenStore.kt // Keychain-based secure token storage
com.steelnotes.platform/
└── PushTokenProvider.kt // APNs device token retrieval
```
```
SteelNotes/
├── Services/
│ ├── SyncService.swift // Swift wrapper around SyncManager
│ └── AuthService.swift // Sign in with Apple UI flow
├── Views/
│ ├── Auth/
│ │ └── SignInView.swift // Sign in with Apple button + onboarding
│ ├── Settings/
│ │ └── SyncSettingsView.swift // Account info, sync toggle, storage usage
│ └── Sync/
│ ├── SyncStatusView.swift // Toolbar sync indicator
│ └── ConflictResolutionView.swift
└── ViewModels/
├── AuthViewModel.swift
└── SyncViewModel.swift
```
server/ — Rust + Axum, monorepo)```
server/
├── Cargo.toml
├── src/
│ ├── main.rs // Axum router setup, Lambda/local server entry
│ ├── routes/
│ │ ├── mod.rs
│ │ ├── auth.rs // POST /auth/login, /auth/refresh
│ │ ├── devices.rs // POST /devices/register
│ │ ├── vault.rs // GET /vault/state, POST /vault/push, pull, delete, export
│ │ └── health.rs // GET /health
│ ├── middleware/
│ │ └── auth.rs // JWT extraction + validation (tower middleware)
│ ├── services/
│ │ ├── mod.rs
│ │ ├── s3.rs // aws-sdk-s3 presigned URL generation
│ │ ├── apple_auth.rs // Validate Apple identity tokens (JWKS)
│ │ ├── push.rs // APNs via a]2 (HTTP/2), FCM future
│ │ └── tokens.rs // JWT creation + refresh token management
│ ├── db/
│ │ ├── mod.rs
│ │ ├── models.rs // User, Device, FileState structs (sqlx::FromRow)
│ │ ├── queries.rs // Typed queries via sqlx
│ │ └── migrations/ // sqlx migrations
│ │ ├── 001_users.sql
│ │ ├── 002_devices.sql
│ │ └── 003_file_state.sql
│ ├── error.rs // AppError → HTTP response mapping
│ └── config.rs // Env-based config (DATABASE_URL, S3_BUCKET, etc.)
├── tests/
│ └── integration/ // API integration tests
├── Dockerfile // Multi-stage: build with rust, run with distroless
└── deploy/
└── terraform/ // AWS infrastructure (Lambda, RDS, S3, API Gateway)
```
Stack: Rust 2024 edition, Axum 0.8, sqlx (Postgres, compile-time checked queries), aws-sdk-s3, jsonwebtoken, tower (middleware)
Deployment: AWS Lambda via cargo-lambda (Rust compiles to a single binary, ~5MB, cold starts ~50ms). Can also run as a standalone binary on ECS Fargate if Lambda latency becomes a concern.
server/ Rust project setup: Cargo.toml, Axum router, config, error handlingusers, devices, file_state tablesPOST /auth/login — Apple identity token validation (JWKS), JWT issuancePOST /auth/refresh — refresh token rotationPOST /devices/register — device + push token registrationcargo-lambda build + deploy pipelineAuthManager, TokenStore (expect/actual for Keychain), SyncClient skeletonSignInView with Sign in with AppleGET /vault/state — full and incremental (?since=)POST /vault/push — version check + presigned upload URLsPOST /vault/push/confirm — update file_state + trigger push notificationsPOST /vault/pull — presigned download URLsPOST /vault/delete — version-checked deletion with tombstonesSyncManager full sync cycleSyncStateDatabase (sync.db schema)ChangeDetector — hash-based local diffingsyncNow()SyncStatusView — toolbar sync indicatorConflictResolver — frontmatter + body split mergeConflictResolutionView — side-by-side diff UIGET /vault/export — zip download of entire vaultSyncSettingsView — account info, storage usage, device listWhen a note is deleted on device A:
1. Device A removes local file, calls POST /vault/delete with base_version
2. API marks file_state.is_deleted = true, sets deleted_at
3. Push notification sent to other devices
4. Other devices pull state, see deletion, remove local file
5. Safety net: If the file was edited locally since last sync, surface as conflict ("Deleted on iPhone, but edited here. Keep?")
6. Tombstones kept for 90 days so stale devices can catch up
sync_state as pending_pushSyncManager runs a full sync cycle429 if quota exceeded; client shows "Storage full" in sync settingsserver/ directory alongside shared/, cli/, and iosApp/.1. Pricing model? Free with limits + paid subscription? Storage cost per user is negligible for text files (~$0.001/month). Real costs are compute (Lambda invocations) and push notifications. Need to decide before Phase 3E.
2. Binary attachments (images, PDFs)? Not yet supported but S3 handles them naturally. When the time comes, large files use multipart upload and the quota system becomes important. Define an attachments/ convention now.
3. E2E encryption timeline? The architecture supports it (API/S3 never read file contents), but key management UX is complex. Defer to post-launch or make it a premium feature?
4. Lambda vs ECS Fargate? Start with Lambda for cost (pay-per-request, $0 at idle). If p99 latency from cold starts becomes a problem at scale, migrate to Fargate with minimum 1 task. The Rust binary runs on either with zero code changes.