agent-api-langfuse-smoke

Files (1)

• SKILL.md

SKILL.md (first lines)

# Agent API Langfuse Smoke

## Purpose

Verify that the Agent API actually ran and that the recursive LLM/tool loop can
be inspected through local Langfuse. Prefer concrete runtime evidence over
code-only conclusions.

This skill is for local Tachyon development with:

- Tachyon API: `http://localhost:50054`
- Langfuse UI: `http://localhost:3000`
- Langfuse project: use the project configured for the local environment.
- Dev operator and user: resolve locally from the seed files; do not copy their IDs into plugin docs.

Langfuse local sign-in, when needed:

```text
email: admin@example.com
password: "${LANGFUSE_LOCAL_PASSWORD}"
```

Load the password from ignored local configuration; never store the value in a
skill or taskdoc.

## What This Confirms

- Agent API endpoint accepts the request.
- A Langfuse session is created or reused.
- `agent.llm_generation` traces are grouped by `sessionId`.
- The child `llm.generation` observation contains the real provider request at
  `input.provider_api_request`.
- Recursive/tool runs create an `agent.execute_tool` trace in the same session.
- Provider, model, usage, tenant, user, and output are visible for debugging.

Langfuse is observability, not a full test runner. Use scenario tests or API
assertions for DB state and business invariants.

## Preconditions

Preferred automated smoke command:

```bash
scripts/test/agent-api-langfuse-smoke.sh
```

Use this script first when it exists. It creates an Agent Session, runs a
minimal Agent API request, waits for Langfuse traces, verifies
`input.provider_api_request`, optionally exercises the tool loop, and prints the
Langfuse session/trace URLs plus saved artifacts. Override local settings with
`TACHYON_API_BASE_URL`, `LANGFUSE_BASE_URL`, `AGENT_MODEL`,
`AGENT_LANGFUSE_TOOL_SMOKE`, `AGENT_LANGFUSE_BUILTIN_TOOLS`, and
`AGENT_LANGFUSE_OUTPUT_DIR` when needed.

Check the local services:

```bash
docker compose ps
curl -sf http://localhost:50054/health
curl -sf http://localhost:3000/api/public/health || true
```

If Langfuse traces are missing, verify the API was started with:

```bash
LANGFUSE_ENABLED=true
OTEL_ENABLED=true
LANGFUSE_ENVIRONMENT=local
TACHYON_AGENT_BILLING_ENABLED=false
```

`docker-compose.langfuse.yml` defaults Agent billing to disabled so local
observability/deep-research smoke tests are not interrupted by dev Stripe
configuration. Set `TACHYON_AGENT_BILLING_ENABLED=true` explicitly only when the
test is meant to verify billing.

For this workspace, a known working dev command is:

```bash
LANGFUSE_ENABLED=true OTEL_ENABLED=true LANGFUSE_ENVIRONMENT=local \
TACHYON_AGENT_BILLING_ENABLED=false \
  docker compose -f compose.yml -f docker-compose.langfuse.yml run -d \
  --service-ports --name tachyon-api-mold-run \
  tachyon-api mold -run cargo run --bin tachyon-api
```

Do not kill or replace an existing API container unless it is clearly stale or
the user asked for a restart.

## Smoke Test: Create Session

Create a fresh Agent Session and keep the session id:

```bash
: "${TACHYON_OPERATOR_ID:?Set this from the local dev seed}"
: "${TACHYON_USER_ID:?Set this from the local dev seed}"
MARKER="langfuse-agent-smoke-$(date +%s)"
SESSION_JSON=$(curl -sS -X POST 'http://localhost:50054/v1/llms/sessions' \
  -H 'Authorization: Bearer dummy-token' \
  -H "x-operator-id: ${TACHYON_OPERATOR_ID}" \
  -H "x-user-id: ${TACHYON_USER_ID}" \
  -H 'Content-Type: application/json' \
  --data-binary "{\"name\":\"${MARKER}\",\"metadata\":{\"marker\":\"${MARKER}\",\"purpose\":\"agent-api-langfuse-smoke\"}}")
SESSION_ID=$(printf '%s' "$SESSION_JSON" | jq -r '.session_id')
printf 'SESSION_ID=%s\nMARKER=%s\n' "$SESSION_ID" "$MARKER"
```

## Smoke Test: LLM Request

Run a minimal Agent API request. Omit `model` first so the tenant's default
agent model is used. If a specific model returns `Model ... is not available`,
fall back to the default model.

```bash
curl -sS -N -X POST "http://localhost:50054/v1/llms/sessions/${SESSION_ID}/agent/execute" \
  -H 'Authorization: Bearer dummy-token' \
  -H "x-operator-id: ${TACHYON_OPERATOR_ID}" \
  -H "x-user-id: ${TACHYON_USER_ID}" \
  -H 'Content-Type: application/json' \

From quantum-box/tachyon-skills, MIT.

More from quantum-box/tachyon-skills