langfuse

Files (14)

• NOTICE.md
• SKILL.md
• scenarios.json · evals/
• langfuse.cjs
• ci-cd.md · references/
• cli.md · references/
• error-analysis.md · references/
• instrumentation.md · references/
• judge-calibration.md · references/
• operator-setup.md · references/
• prompt-migration.md · references/
• sdk-upgrade.md · references/
• skill-feedback.md · references/
• user-feedback.md · references/

SKILL.md (first lines)

# Langfuse

This skill helps you use Langfuse effectively across all common workflows:
instrumenting applications, migrating prompts, debugging traces, accessing data,
and evaluating outputs.

> **HybridClaw integration.** This is the official Langfuse skill
> ([github.com/langfuse/skills](https://github.com/langfuse/skills), MIT)
> adapted for HybridClaw. Two things differ from the upstream skill:
>
> 1. **Credentials never leave the gateway.** Do not export
>    `LANGFUSE_PUBLIC_KEY` / `LANGFUSE_SECRET_KEY`, run `npx langfuse-cli`, or
>    paste keys anywhere. Store them once in the runtime stores (below); the
>    gateway injects them server-side.
> 2. **Data access goes through `langfuse.cjs`**, not the Langfuse CLI. The
>    helper builds each REST request and sends it through the HybridClaw gateway,
>    which resolves `Authorization: Basic <secret:LANGFUSE_BASIC_AUTH>` and the
>    `<env:LANGFUSE_HOST>` base URL. Wherever a reference says to run
>    `langfuse-cli` or `curl -H "Authorization: Basic $AUTH"`, use the helper
>    instead. Documentation retrieval (section 2) is unchanged.

## HybridClaw setup

The helper never sees credentials. Store two values once. For secrets, use this
order:

1. Browser admin: open the active HybridClaw admin URL ending in `/admin/secrets`.
2. Browser `/chat` or TUI fallback.
3. Local console fallback.

1. `LANGFUSE_BASIC_AUTH` — base64 of `public-key:secret-key`:
   `/secret set LANGFUSE_BASIC_AUTH "<base64-public-colon-secret>"`
2. `LANGFUSE_HOST` — your Langfuse base URL:
   `/env set LANGFUSE_HOST https://cloud.langfuse.com` (use
   `https://us.cloud.langfuse.com` for US, `https://jp.cloud.langfuse.com` for
   JP, or your self-hosted origin).

Local console fallback: `hybridclaw secret set LANGFUSE_BASIC_AUTH "<...>"`
and `hybridclaw env set LANGFUSE_HOST https://cloud.langfuse.com`.

See [references/operator-setup.md](references/operator-setup.md) for key scope,
host selection, autonomy defaults, and network-policy notes.

## Core Principles

Follow these principles for ALL Langfuse work:

1. **Documentation first**: never implement from memory. Langfuse updates
   frequently — fetch current docs (section 2) before writing instrumentation or
   SDK code.
2. **Helper for data access**: use `langfuse.cjs` (gateway + SecretRef) when
   querying or modifying Langfuse data. It owns endpoints, methods, bodies,
   stakes tiers, host, and the Basic auth placeholder.
3. **Best practices by use case**: check the relevant reference below before
   implementing.
4. **Use latest Langfuse versions**: unless the user says otherwise, target the
   latest Langfuse SDKs/APIs.

## Use-case references

- instrumenting an existing function/application:
  [references/instrumentation.md](references/instrumentation.md)
- migrating prompts from a codebase into Langfuse:
  [references/prompt-migration.md](references/prompt-migration.md)
- capturing user feedback (thumbs, ratings, implicit signals) as scores:
  [references/user-feedback.md](references/user-feedback.md)
- systematic error analysis — reading traces, building a failure taxonomy,
  deciding what to fix: [references/error-analysis.md](references/error-analysis.md)
- judge calibration (LLM-as-a-Judge reliability, accuracy checks, confusion
  matrices, metric ingestion):
  [references/judge-calibration.md](references/judge-calibration.md)
- upgrading or migrating Langfuse SDKs:
  [references/sdk-upgrade.md](references/sdk-upgrade.md)
- CI/CD experiment gates with `langfuse/experiment-action`:
  [references/ci-cd.md](references/ci-cd.md)
- raw Langfuse REST/CLI semantics (endpoints, pagination, v2 vs legacy):
  [references/cli.md](references/cli.md)
- HybridClaw credential, host, autonomy, and network-policy setup:
  [references/operator-setup.md](references/operator-setup.md)
- submitting feedback about this skill:
  [references/skill-feedback.md](references/skill-feedback.md)

## 1. Langfuse data access (HybridClaw gateway helper)

`langfuse.cjs` is the API wrapper. Do not handcraft Langfuse API URLs, JSON
bodies, tiers, host, or the Basic auth header from memory.

```bash
node skills/langfuse/langfuse.cjs --help
```

- **plan** classifies a natural-language request into an operation + tier:
  `node skills/langfuse/langfuse.cjs --format json plan "average eval score this week"`
- **run** executes a live request through the gateway (the gateway injects the
  Basic auth header):
  `node skills/langfuse/langfuse.cjs --format json run list-traces --user-id alice --limit 50`
- **http-request** emits the gateway-ready payload without calling Langfuse —
  use it for dry-run inspection or runtimes without helper gateway access.

Read examples (green):

```bash
node skills/langfuse/langfuse.cjs --format json run get-trace --trace-id abc123
node skills/langfuse/langfuse.cjs --format json run list-observations --type GENERATION --trace-id abc123
node skills/langfuse/langfuse.cjs --format json run list-scores --name quality
node skills/langfuse/langfuse.cjs --format json run get-prompt --prompt-name support-reply --label production
node skills/langfuse/langfuse.cjs --format json run metrics --query '{"view":"traces","metrics":[{"measure":"count","aggregation":"count"}]}'
```

Guarded write examples (amber — only after an explicit operator grant):

```bash
node skills/langfuse/langfuse.cjs --format json run create-score \
  --trace-id abc123 --name quality --value 0.8 --data-type NUMERIC --comment "reviewed" --operator-grant
node skills/langfuse/langfuse.cjs --format json run create-prompt \
  --name summarizer --type text --prompt "Summarize: {{input}}" --label production --operator-grant
```

Select region or self-hosted host explicitly (otherwise `<env:LANGFUSE_HOST>`):

From HybridAIOne/hybridclaw, MIT.

More from HybridAIOne/hybridclaw