Goal. Replace plaintext vault paths in file_state and S3 keys with
HMAC + AES-GCM blobs so an operator running SELECT * FROM file_state or
listing the S3 bucket sees opaque hex / base64 instead of
notes/divorce-journal.md.
Threat model. Protects against casual / accidental exposure. An
operator with both DB access AND the Secrets Manager keys can still
reverse the mapping — same posture as password hashing. True E2E privacy
(operator can't read even when motivated) requires client-held keys and
breaks the AI features that read note bodies. Out of scope here.
Why two secrets.
PATH_HMAC_KEY → deterministic HMAC-SHA256 for indexed lookups.PATH_ENC_KEY → randomized AES-256-GCM for round-trip (the wireprotocol still ships plaintext path to clients; server decrypts on
read).
Independent secrets so they can be rotated separately.
The change touches the storage layer of every sync operation. Rolled out
in five phases instead of one big-bang diff. Phases 1 + 2 are landed.
adds nullable path_hash TEXT + path_ciphertext TEXT and a partial
unique index on (user_id, path_hash) WHERE path_hash IS NOT NULL.
sqlx::migrate!.is the helper. PathCrypto::from_hex(hmac_hex, enc_hex) -> Option<Self>:
empty strings → None. 7 unit tests cover determinism, cross-user
salting, AEAD round-trip, fresh-nonce-per-call, validation errors.
PATH_HMAC_KEY /PATH_ENC_KEY (default "").
path_crypto: Option<PathCrypto> on AppState. None when secrets
are absent → existing behavior unchanged (local dev safe).
vault_push_confirm (~line 154).
(~line 488). Same change also scrubbed the tracing::info! log line
from path=… to path_hash=… so CloudWatch stops echoing plaintext
once secrets are set.
queries::upsert_file_state now takes two extra Option<&str> args(path_hash, path_ciphertext). COALESCE(EXCLUDED.path_hash, file_state.path_hash)
in the upsert means an unhashed update can't blank out a previously
hashed row.
db::models::FileState carries path_hash: Option<String> +path_ciphertext: Option<String> for SELECT * callers.
Current state on disk. Files modified but not committed as of
handoff. cargo check --features lambda passes. `cargo test --lib
services::path_crypto` → 7/7 pass. Server has not been deployed since
this work — the Lambdas in prod don't have the new columns or dual-write
code yet.
Prereq: Phases 1 + 2 deployed AND PATH_HMAC_KEY / PATH_ENC_KEY set
in Lambda env. Generate the secrets:
```bash
openssl rand -hex 32 # PATH_HMAC_KEY
openssl rand -hex 32 # PATH_ENC_KEY
```
Add to AWS Secrets Manager and reference from
server/deploy/terraform/main.tf (the
aws_lambda_function.environment.variables block on both api and
worker). Then terraform apply + ./deploy.sh.
The backfill script is not yet written. It must:
1. SELECT user_id, path, path_hash IS NULL AS needs FROM file_state — find
rows missing hash/ciphertext (skip rows where backfill already ran so it's
idempotent).
2. For each row:
path_hash = PathCrypto::hash_path(user_id, path).path_ciphertext = PathCrypto::encrypt_path(path).aws s3 cp the object from <user_id>/<path> to <user_id>/<path_hash>.aws s3 rm the old key (only after the copy succeeds).user_id = $3 AND path = $4`.
3. Rerun until SELECT COUNT(*) FROM file_state WHERE path_hash IS NULL
returns 0.
Suggested form: a bin/backfill_paths.rs that takes the keys via env and
runs inside Lambda for IAM-free S3 access, OR a local script using AWS
CLI + psql. For the current data volume (one developer + a few testers)
a local script is fine.
Gotchas:
delete_all_under_user SAFE since it sweeps thewhole <user_id>/ prefix regardless of hash.
worker.rs AI-result write uses `path = "<user>/.steel/ai-results/<capture>.json"`-style paths. The hash will collide-safely with
anything else (it's all HMAC), but verify by inspecting a sample row
after backfill.
After backfill, flip every read to use path_hash:
get_file_state(user_id, path) → hash internally, lookup bypath_hash. Or add a new get_file_state_by_hash and migrate
callers; cleaner.
mark_file_deleted(user_id, path) → same.get_vault_state returns FileState rows — adjust SELECT to aliaspath_ciphertext back into path after decrypt, OR push decrypt to
the route layer.
object_key(user_id, path) (~line 34) — change to hash the path
internally. Needs PathCrypto injected into S3Service (currently
doesn't hold any extra state). Easiest: pass crypto on construction in
lib.rs.
vault_state GET — when streaming RemoteFileState rows back to the
client, replace f.path with crypto.decrypt_path(&f.path_ciphertext).
Same for DeletedFile.path. Wire protocol stays plaintext-path-based.
vault_push lookup loop (~line 88) — hash before query.
This is the largest single phase. Plan ~2-3 hours.
```sql
-- 015_drop_plaintext_path.sql
ALTER TABLE file_state ALTER COLUMN path_hash SET NOT NULL;
ALTER TABLE file_state ALTER COLUMN path_ciphertext SET NOT NULL;
ALTER TABLE file_state DROP CONSTRAINT file_state_pkey;
ALTER TABLE file_state ADD PRIMARY KEY (user_id, path_hash);
ALTER TABLE file_state DROP COLUMN path;
DROP INDEX idx_file_state_user_path_hash; -- redundant with the new PK
```
Also at this phase:
path field from db::models::FileState.path arg from upsert_file_state (only the hash mattersfor the unique row identity; the plaintext is in ciphertext).
tracing::*! lines that includef.path or &path — scrub.
```
M server/Cargo.toml # +hmac +aes-gcm +rand
A server/src/db/migrations/014_path_anonymization.sql
A server/src/services/path_crypto.rs # + tests
M server/src/services/mod.rs # export PathCrypto
M server/src/config.rs # PATH_HMAC_KEY / PATH_ENC_KEY
M server/src/lib.rs # AppState.path_crypto
M server/src/db/models.rs # FileState +2 nullable fields
M server/src/db/queries.rs # upsert_file_state +2 args
M server/src/routes/vault.rs # dual-write in vault_push_confirm
M server/src/worker.rs # dual-write + log scrub
```
```bash
cd server
export CARGO_TARGET_DIR=$HOME/.cargo-target
cargo check --features lambda # passes
cargo test --lib services::path_crypto # 7 passed
```
1. git status — confirm the files above are still dirty (or committed —
I left them uncommitted at handoff; the previous commit on the branch
is 2049ebe "Billing: 7-day trial + monthly subscription").
2. Commit Phase 1+2 (suggested message: "Path anonymization: dual-write
phase 1+2; reads still plaintext").
3. Generate secrets (commands above), add to Secrets Manager + Terraform,
terraform apply, ./deploy.sh. Verify with one push that new
file_state rows have non-null path_hash / path_ciphertext.
4. Write + run the backfill script (Phase 3). Verify zero NULL rows.
5. Phase 4 read switch — biggest chunk. Test on staging if you set one
up; otherwise be careful, sync break = user-visible data loss.
6. Phase 5 schema cleanup once Phase 4 has been stable for a day.