mfs-find

Files (22)

• SKILL.md
• bigquery.md · reference/connectors/
• discord.md · reference/connectors/
• feishu.md · reference/connectors/
• file.md · reference/connectors/
• gdrive.md · reference/connectors/
• github.md · reference/connectors/
• gmail.md · reference/connectors/
• hubspot.md · reference/connectors/
• jira.md · reference/connectors/
• linear.md · reference/connectors/
• mongo.md · reference/connectors/
• mysql.md · reference/connectors/
• notion.md · reference/connectors/
• postgres.md · reference/connectors/
• s3.md · reference/connectors/
• slack.md · reference/connectors/
• snowflake.md · reference/connectors/
• web.md · reference/connectors/
• zendesk.md · reference/connectors/
• error-codes.md · reference/
• json-envelope.md · reference/

SKILL.md (first lines)

# MFS — find / read across configured sources

## 1. What MFS is

A retrieval layer that exposes many kinds of content as a unified path
tree and makes that tree searchable through one hybrid index:

- **One CLI (`mfs`), one mental model.** Local dir, Postgres, GitHub repo,
  Slack workspace, S3 bucket, BigQuery dataset — all addressed as paths
  under their `<scheme>://` URI. Same verbs everywhere: `ls / tree / cat /
  head / tail / grep / search / export`.
- **One hybrid index.** Dense vectors (semantic) + BM25 (keyword) fused
  per query — covers conceptual recall and exact-token recall in one call.
- **POSIX-style locators.** Every search hit carries a `locator` that
  reopens the exact unit: `{"lines":[s,e]}` for text/code, a PK dict for
  rows/issues/threads.

## 2. When to use MFS — and when NOT to

| Situation | Use MFS? |
|---|---|
| 1000+ files / rows / pages, you don't know where the answer is | ✅ |
| Cross-source question ("any past tickets / commits / RFCs about X") | ✅ `--all` |
| Concept-style query that won't match literally | ✅ `--mode semantic` |
| You already know the exact file + roughly where to look | ❌ plain `cat`/`grep` |
| Exact identifier / error code in 5 files you can list | ❌ plain `grep`/`rg` |
| Real-time tailing of a live log | ❌ index lags ingest |
| The source isn't in MFS yet | wrong skill, use `mfs-ingest` to register first |

**Rule:** use the smallest tool that answers the question. MFS pays off
when the scope is too big for `rg`.

**Borderline — ASK the user:**

| Ask | Likely answer | Why |
|---|---|---|
| "Summarise these 10 PDFs" | ✅ `mfs search` + `cat --peek` per hit | each PDF gets a `converted_md` artifact + searchable chunks |
| "Find similar tickets to this one" | ✅ paste the ticket text as the search query | semantic over `row_text` does similarity matching |
| "Watch for new slack messages" | ❌ no `watch` capability; use Slack's API | index lags ingest |
| "Look up user 12345" | ❌ `mfs cat <source> --locator '{"id":12345}'` directly (skip search) | one-record-by-id doesn't need ranking |

## 3. Pre-flight — confirm the source is indexed

Before running any query, especially on cross-source asks:

```bash
mfs status                  # server up? any connectors registered?
mfs connector inspect <uri> # this connector's object/job summary
mfs ls <uri> --json         # per-entry capabilities + indexable / search_status
```

- Server unreachable → tell user to start it (`mfs serve start` if
  self-hosted), or this skill can't proceed.
- `connectors` empty → user hasn't ingested anything yet. **Redirect to
  `mfs-ingest`** — don't try to search nothing.
- `search_status: unavailable` for the target URI → only `grep` / `ls` /
  `cat` work; offer those or redirect to `mfs-ingest` for a re-sync.
- `building` → sync in flight; fall back to `mfs grep` (works without an
  index) until done.
- `partial` → recall incomplete but usable; flag the caveat to the user.

## 4. The core workflow: search → locate → browse

```
        search                 locate                browse
  ┌──────────────────┐   ┌──────────────────┐   ┌─────────────────────┐
  │  semantic + BM25 │ → │ result has lines │ → │  cat --range / cat  │
  │  finds candidates│   │  or a locator    │   │  --peek to confirm  │
  └──────────────────┘   └──────────────────┘   └─────────────────────┘
```

On large corpora this loop is the whole point: read only the part that
matters. On small corpora it's still fine, just lighter.

Concrete:

1. **Search:**
   ```bash
   mfs search "<what the user actually wants>" <path-or-uri> --top-k 10
   ```
2. **Locate** — every hit's envelope carries `locator`:
   - text/code → `{"lines":[start,end]}` → `mfs cat <source> --range start:end`
   - structured (row/issue/thread) → PK dict → `mfs cat <source> --locator '{...}'`
   - once-per-object (dir/schema summary, image VLM) → `null` → `mfs cat <source>`
3. **Browse** — verify only what's needed:
   ```bash
   mfs cat --peek <file>       # outline (headings / function signatures)
   mfs cat --skim <file>       # peek + one-line summaries per section
   mfs head -n 20 <uri>        # first records of a structured object
   mfs tree <uri> -L 2         # subtree shape
   ```

## 5. Index requirement rules of thumb

- `mfs search` **requires** an index.
- `mfs grep` works **without** — pushdown → BM25 → linear scan fallback.
- `mfs ls / tree / cat / head / tail` browse **without** an index.

## 6. Search modes

`mfs search` defaults to `hybrid`. Override only when you know why.

| `--mode` | Mechanic | When |
|---|---|---|
| **`hybrid`** *(default)* | dense + BM25 fused with RRF | almost always |
| `semantic` | dense only | conceptual query, wording won't match literally |
| `keyword` | BM25 only | exact-term (config key, error code) without semantic drift |

Other useful flags:

- `--top-k N` — default 10; raise to 20-30 on a weak first round.
- `--all` — search every registered connector. Otherwise scope to a path/URI prefix.
- `--kind <list>` — restrict chunk kinds (`row_text`, `thread_aggregate`,
  `body`, `summary`, `vlm_description`, …).
- `--collapse` — keep only the top-scoring chunk per object; later chunks
  from the same source are dropped, not merged. Recall stays as-is (the
  query still hits the same candidates), but the visible result count can
  fall below `--top-k` — collapse is a post-filter, not a re-rank. If you
  want N distinct objects, raise `--top-k` (e.g. `--top-k 30 --collapse`).

From zilliztech/mfs, Apache-2.0.

More from zilliztech/mfs