Slack → vault sync

One JSON file per channel, incremental. extracted/slack/<channel>/conversation.json is the deliverable: messages in chronological order, replies nested under their parent, attachments downloaded alongside and referenced by local path.

extracted/slack/
  channels.json       # roster + per-channel watermark + workspace fingerprint
  users.json           # id → name/email cache
  <channel>/
    conversation.json  # ⬅ the deliverable
    files/              # attachments + MANIFEST.json (+ MISSING.md if something is missing)
    INDEX.md             # human-readable index, regenerated

conversation.json is the only archive of the conversation: don’t create files for individual messages, threads, or pages. Binary attachments stay in files/ because they are distinct documents and are referenced from both the JSON and MANIFEST.json.

Two paths. With a Slack token (preferred) the script does everything: enumerate, paginate, follow threads, download files. Without a token you go through the MCP connector, which is slower and does not enumerate. Two modes: backfill (first pass, everything) and update (only what’s new).

extracted/slack/ is source material, not the source of truth for content: facts belong in your project’s own topic/record store, routed there by whatever extract/ingest step your project uses. Keep the narrative source of truth for watermarks in one place in your project (e.g. a slack-channels.md status doc); channels.json is its machine mirror.

---

Path A — Slack API (default)

User token (xoxp-…) in $SLACK_TOKEN or ~/.config/slack-credentials/credentials.json ({"token": "xoxp-…"}, chmod 600). Scopes: channels:history, channels:read, groups:history, groups:read, im:history, mpim:history, users:read, files:read. The token is never printed or written into the archive.

S=".claude/skills/slack-vault-sync/scripts/slack_sync.py"

uv run $S fingerprint          # auth.test → registers team/team_id in the archive
uv run $S discover             # conversations.list → TRUE enumeration of the workspace
uv run $S backfill --all       # FIRST PASS: full history + threads + attachments
uv run $S update --all         # SUBSEQUENT PASSES: only what's new, from the watermark
uv run $S status --rows        # rows ready to paste into your channel-status doc

backfill and update handle everything on their own: paginated history, conversations.replies for every thread, attachment downloads with the auth header (no base64 in context), merge by ts, watermark, INDEX.md. They write directly into a single conversation.json per channel and produce no raw payloads. Re-running them is safe: the merge neither duplicates nor deletes.

Before launching backfill --all, check the output of discover: it flags channels you are not a member of (Slack will deny history — you need to join them first) and estimates the work. A large workspace takes minutes and many calls; --no-files defers attachments to a second pass, --channel <id> does one channel at a time.

The trade-off to know: replies on old threads

conversations.history does not return replies, and a new reply on an old thread does not make the parent reappear in history. So update re-checks replies on already-archived threads up to --thread-lookback-days (default 30). A reply on a thread older than that is missed until you rerun with a wider window:

uv run $S update --all --thread-lookback-days 120

If the client resurrects old threads (it happens), raise the default or periodically run an update --thread-lookback-days 0 --channel <id> after a backfill --force.

---

Path B — MCP (when there’s no token)

Same output, but the agent acts as the transport. It costs context and does not enumerate.

1. Workspace: slack_read_user_profile → fingerprint --team … --user-id …. If the fingerprint doesn’t match, stop: same-named channels can exist in different workspaces.
2. Channels: slack_search_channels is fuzzy search, not a listing — measured on 2026-08-12: query="a" → 3 channels, query="e" → 0, query="*" → 0. Use multiple queries in union (single vowels, known names, domain terms, company name), cross-check with slack_search_public_and_private (after:<date>, sort=timestamp), and confirm against the Slack sidebar. Then roster-merge --stdin with a JSON [{id,name,type,purpose}].
3. Messages: slack_read_channel (oldest=<watermark>, limit=100, paginate with cursor until pagination_info says there’s no more) and, for every thread, slack_read_thread. Pass every response verbatim to capture, which normalizes it and merges it straight into conversation.json:

uv run $S capture --channel <id> --page <n> <<'SLACK_RAW'
<MCP response pasted verbatim>
SLACK_RAW

uv run $S capture --channel <id> --kind thread --thread-ts <parent_ts> --page 1 <<'SLACK_RAW'
<slack_read_thread response>
SLACK_RAW

4. If it prints ⚠️ unrecognized metadata, those are fields the connector emits that the parser doesn’t know: they stay in the message body (nothing is lost), add them to KNOWN_META and re-run the same response through capture — it’s idempotent. parse --all exists only as a compatibility migration for old _raw/ payloads that may already be present, and removes them after ingestion.
5. Attachments: slack_read_file returns base64 for binaries, which flows through context. Download only the small ones (file-add --b64-file), declare the rest (file-missing --reason … --permalink … → ends up in files/MISSING.md, recovered from the UI). An attachment that is undownloaded but declared is fine; one that is forgotten is not.
6. Manual watermark: watermark --channel <id> --ts last --note "how far".

---

Wrap-up (always, either path)

1. status --rows → update your channel-status doc’s table, including channels that came back empty (that’s the point of the placeholder row);
2. route the content: answers to open records → your ingest step; new requirements → your extract step;
3. run your project’s checks GREEN before committing.

Commands

Command	What it does
fingerprint	registers/verifies the archive’s workspace — first step, always
discover	[API] enumerates channels via conversations.list
backfill	[API] first pass: full history + threads + attachments
update	[API] incremental from the watermark
roster-merge · capture · parse	[MCP] manual roster · normalize+merge · legacy migration
file-add / file-missing	manually recovered attachment / declared missing
index · watermark · status	regenerate INDEX.md · move the watermark · state and rows

Verified and unverified tool behavior: references/mcp-notes.md.

Mistakes not to make

Mistake	Why it burns you
Hand-writing conversation.json or summarizing messages	the archive diverges from the real messages with nobody noticing; it must only be produced by backfill/update/capture
backfill instead of update for routine runs	re-reads all of history and wastes rate limit (though the merge won’t duplicate)
update with the default lookback on old threads	late replies stay out: raise --thread-lookback-days
Trusting slack_search_channels for the channel list	it’s fuzzy search: new channels stay invisible for days
Syncing without fingerprint	you pour a different workspace into the client’s archive
Moving the watermark by hand “by eye”	you skip messages never read; backfill/update move it themselves to the last archived ts
Not updating empty channels in your channel-status doc	the next pass re-reads them from scratch