OpenMark

Install

Works with Claude Code, VS Code, Cursor, Windsurf. Read the repository before you give any tool access to your data.

npx agentmarket install connector/openmark-ai-openmark

About

No reviews yet. Installed it? Reviews come from verified installers only. How reviews work

Permissions

What this server can do and what it asks for, from its tool list and install config. Checked Oct 25, 2026.

Tools

8 read · 1 write · 0 delete

Secrets

1 (API keys, tokens or passwords)

Install scripts

Nothing to install (hosted)

Runs as

Hosted (remote endpoint)

Tools (9)

As the server listed them on Oct 25, 2026. Descriptions are the server’s own words; hidden characters are shown as [U+…].

• estimate_costEstimate a run's cost

  Credits a run_benchmark call would use, before spending anything: the expected total, the split per model, the balance needed to start, the worst case, and the user's balance. Same inputs and run settings as run_benchmark. Use it to drop an expensive model, or to compare settings (e.g. max_tokens), before running.

• get_accountAccount, credits and plans

  The user's plan, credit balance and limits, and what a credit pack, Pro or Expert would unlock. Use it before large runs, or when a run is refused for credits or limits.

• get_resultsGet benchmark results

  Status of a benchmark job, and once completed the ranked results (score, stability, cost per run, latency), the tests each model missed with its answer, a recommendation for the chosen objective and quick picks for the others.

• get_task_formatTask YAML format

  How to write an OpenMark task: the YAML structure, all 18 scoring modes, and tips. Call this before writing a task. Pass a topic for the full detail on one subject (the app's Task Guide): fields, scoring, composite, chaining, documents, examples, best_practices, or a scoring mode name.

• list_modelsList models

  Current OpenMark model ids with list prices ($ per 1M tokens), as text and as a structured list. Use these ids in run_benchmark. Append @low, @medium or @high to a reasoning model id to set its reasoning effort.

• prepare_attachmentsPrepare file attachments

  Get an upload_uid and a one-hour upload link (no API key needed) for images, PDFs or text documents, with the exact curl command. Upload the files, reference them by file name in each test's attachments list, then pass the upload_uid to estimate_cost / run_benchmark.

• run_benchmarkRun a benchmark

  Save a task (YAML, see get_task_format) to the user's OpenMark account and run it on the given models with real API calls. Returns a job_id and the expected cost; poll get_results. Spends the user's credits; pass max_credits to only start within a budget. Optional run settings match the app's Configure & Run panel (preset, stability_runs, max_tokens, temperature_discovery, temp_preference, timeout_profile, custom_timeout, fail_fast).

• validate_taskCheck a task

  Check a task YAML the way OpenMark will before running it: structure, scoring settings, and the test limit of the user's plan. Free, saves nothing. Returns the problems to fix and warnings worth a look.

• watch_taskWatch a task for better models

  Watch the task of a completed job: when new or cheaper models ship, OpenMark re-tests up to 4 of them next to the user's pick and emails the user only when one beats it. Each check uses the user's credits at the normal rate.

Signals

Last release

Oct 18, 2026 (10 days ago)

Details

Registry name

ai.openmark/openmark

Version

1.0.0

Auth

Bearer token

License

License unknown

Website

openmark.ai/mcp

First seen

Oct 24, 2026

Last checked

Oct 24, 2026

Source

Official MCP Registry