← Back to Plan

Path Anonymization

Path Anonymization — Tier 1 (in-flight, handoff)

Goal. Replace plaintext vault paths in file_state and S3 keys with

HMAC + AES-GCM blobs so an operator running SELECT * FROM file_state or

listing the S3 bucket sees opaque hex / base64 instead of

notes/divorce-journal.md.

Threat model. Protects against casual / accidental exposure. An

operator with both DB access AND the Secrets Manager keys can still

reverse the mapping — same posture as password hashing. True E2E privacy

(operator can't read even when motivated) requires client-held keys and

breaks the AI features that read note bodies. Out of scope here.

Why two secrets.

  • PATH_HMAC_KEY → deterministic HMAC-SHA256 for indexed lookups.
  • PATH_ENC_KEY → randomized AES-256-GCM for round-trip (the wire
  • protocol still ships plaintext path to clients; server decrypts on

    read).

    Independent secrets so they can be rotated separately.

    Rollout phases

    The change touches the storage layer of every sync operation. Rolled out

    in five phases instead of one big-bang diff. Phases 1 + 2 are landed.

    ✅ Phase 1 — Schema (additive)

  • server/src/db/migrations/014_path_anonymization.sql
  • adds nullable path_hash TEXT + path_ciphertext TEXT and a partial

    unique index on (user_id, path_hash) WHERE path_hash IS NOT NULL.

  • Auto-applies on Lambda cold start via sqlx::migrate!.
  • ✅ Phase 2 — Dual-write (no-op without secrets)

  • server/src/services/path_crypto.rs
  • is the helper. PathCrypto::from_hex(hmac_hex, enc_hex) -> Option<Self>:

    empty strings → None. 7 unit tests cover determinism, cross-user

    salting, AEAD round-trip, fresh-nonce-per-call, validation errors.

  • server/src/config.rs reads PATH_HMAC_KEY /
  • PATH_ENC_KEY (default "").

  • server/src/lib.rs puts
  • path_crypto: Option<PathCrypto> on AppState. None when secrets

    are absent → existing behavior unchanged (local dev safe).

  • Dual-write sites:
  • server/src/routes/vault.rs
  • vault_push_confirm (~line 154).

  • server/src/worker.rs AI-result write
  • (~line 488). Same change also scrubbed the tracing::info! log line

    from path=… to path_hash=… so CloudWatch stops echoing plaintext

    once secrets are set.

  • queries::upsert_file_state now takes two extra Option<&str> args
  • (path_hash, path_ciphertext). COALESCE(EXCLUDED.path_hash, file_state.path_hash)

    in the upsert means an unhashed update can't blank out a previously

    hashed row.

  • db::models::FileState carries path_hash: Option<String> +
  • path_ciphertext: Option<String> for SELECT * callers.

    Current state on disk. Files modified but not committed as of

    handoff. cargo check --features lambda passes. `cargo test --lib

    services::path_crypto` → 7/7 pass. Server has not been deployed since

    this work — the Lambdas in prod don't have the new columns or dual-write

    code yet.

    🔲 Phase 3 — Backfill (your action when ready)

    Prereq: Phases 1 + 2 deployed AND PATH_HMAC_KEY / PATH_ENC_KEY set

    in Lambda env. Generate the secrets:

    ```bash

    openssl rand -hex 32 # PATH_HMAC_KEY

    openssl rand -hex 32 # PATH_ENC_KEY

    ```

    Add to AWS Secrets Manager and reference from

    server/deploy/terraform/main.tf (the

    aws_lambda_function.environment.variables block on both api and

    worker). Then terraform apply + ./deploy.sh.

    The backfill script is not yet written. It must:

    1. SELECT user_id, path, path_hash IS NULL AS needs FROM file_state — find

    rows missing hash/ciphertext (skip rows where backfill already ran so it's

    idempotent).

    2. For each row:

  • Compute path_hash = PathCrypto::hash_path(user_id, path).
  • Compute path_ciphertext = PathCrypto::encrypt_path(path).
  • aws s3 cp the object from <user_id>/<path> to <user_id>/<path_hash>.
  • aws s3 rm the old key (only after the copy succeeds).
  • `UPDATE file_state SET path_hash = $1, path_ciphertext = $2 WHERE
  • user_id = $3 AND path = $4`.

    3. Rerun until SELECT COUNT(*) FROM file_state WHERE path_hash IS NULL

    returns 0.

    Suggested form: a bin/backfill_paths.rs that takes the keys via env and

    runs inside Lambda for IAM-free S3 access, OR a local script using AWS

    CLI + psql. For the current data volume (one developer + a few testers)

    a local script is fine.

    Gotchas:

  • Run during a quiet window — concurrent writes will race the copy/delete.
  • Mark account-deletion's delete_all_under_user SAFE since it sweeps the
  • whole <user_id>/ prefix regardless of hash.

  • The worker.rs AI-result write uses `path = "<user>/.steel/ai-results/
  • <capture>.json"`-style paths. The hash will collide-safely with

    anything else (it's all HMAC), but verify by inspecting a sample row

    after backfill.

    🔲 Phase 4 — Switch reads

    After backfill, flip every read to use path_hash:

  • server/src/db/queries.rs:
  • get_file_state(user_id, path) → hash internally, lookup by
  • path_hash. Or add a new get_file_state_by_hash and migrate

    callers; cleaner.

  • mark_file_deleted(user_id, path) → same.
  • get_vault_state returns FileState rows — adjust SELECT to alias
  • path_ciphertext back into path after decrypt, OR push decrypt to

    the route layer.

  • server/src/services/s3.rs
  • object_key(user_id, path) (~line 34) — change to hash the path

    internally. Needs PathCrypto injected into S3Service (currently

    doesn't hold any extra state). Easiest: pass crypto on construction in

    lib.rs.

  • server/src/routes/vault.rs
  • vault_state GET — when streaming RemoteFileState rows back to the

    client, replace f.path with crypto.decrypt_path(&f.path_ciphertext).

    Same for DeletedFile.path. Wire protocol stays plaintext-path-based.

  • server/src/routes/vault.rs
  • vault_push lookup loop (~line 88) — hash before query.

    This is the largest single phase. Plan ~2-3 hours.

    🔲 Phase 5 — Drop plaintext column

    ```sql

    -- 015_drop_plaintext_path.sql

    ALTER TABLE file_state ALTER COLUMN path_hash SET NOT NULL;

    ALTER TABLE file_state ALTER COLUMN path_ciphertext SET NOT NULL;

    ALTER TABLE file_state DROP CONSTRAINT file_state_pkey;

    ALTER TABLE file_state ADD PRIMARY KEY (user_id, path_hash);

    ALTER TABLE file_state DROP COLUMN path;

    DROP INDEX idx_file_state_user_path_hash; -- redundant with the new PK

    ```

    Also at this phase:

  • Drop the path field from db::models::FileState.
  • Remove the path arg from upsert_file_state (only the hash matters
  • for the unique row identity; the plaintext is in ciphertext).

  • Audit one more time for stray tracing::*! lines that include
  • f.path or &path — scrub.

    Files touched (phase 1 + 2)

    ```

    M server/Cargo.toml # +hmac +aes-gcm +rand

    A server/src/db/migrations/014_path_anonymization.sql

    A server/src/services/path_crypto.rs # + tests

    M server/src/services/mod.rs # export PathCrypto

    M server/src/config.rs # PATH_HMAC_KEY / PATH_ENC_KEY

    M server/src/lib.rs # AppState.path_crypto

    M server/src/db/models.rs # FileState +2 nullable fields

    M server/src/db/queries.rs # upsert_file_state +2 args

    M server/src/routes/vault.rs # dual-write in vault_push_confirm

    M server/src/worker.rs # dual-write + log scrub

    ```

    Verification

    ```bash

    cd server

    export CARGO_TARGET_DIR=$HOME/.cargo-target

    cargo check --features lambda # passes

    cargo test --lib services::path_crypto # 7 passed

    ```

    Resume checklist

    1. git status — confirm the files above are still dirty (or committed —

    I left them uncommitted at handoff; the previous commit on the branch

    is 2049ebe "Billing: 7-day trial + monthly subscription").

    2. Commit Phase 1+2 (suggested message: "Path anonymization: dual-write

    phase 1+2; reads still plaintext").

    3. Generate secrets (commands above), add to Secrets Manager + Terraform,

    terraform apply, ./deploy.sh. Verify with one push that new

    file_state rows have non-null path_hash / path_ciphertext.

    4. Write + run the backfill script (Phase 3). Verify zero NULL rows.

    5. Phase 4 read switch — biggest chunk. Test on staging if you set one

    up; otherwise be careful, sync break = user-visible data loss.

    6. Phase 5 schema cleanup once Phase 4 has been stable for a day.