Terradev

Get workflow status.

Test connection to LangGraph service.

Send a message to a Letta agent.

Create a new stateful Letta agent.

Delete a Letta agent.

List Letta agents.

Teach a Letta agent a durable fact.

Show the state of a Letta agent.

List all known agent fleets.

Store a memory in Mem0 for an agent or user.

Configure Mem0 credentials and defaults.

Delete a memory by ID.

Delete all memories matching an entity scope.

Get a single memory by ID.

List memories for an entity scope.

Search agent/user memories.

Test connection to Mem0.

Update a memory by ID.

Plan a heterogeneous agent fleet without provisioning.

Scale a single fleet tier up or down without affecting other tiers.

Attach a skill.md to a Letta agent as a durable memory block.

Create a skill.md template for an agent.

Show live status of a fleet — tier health, KV hit rate, queue depth, cost.

Terminate all fleet instances and remove fleet state.

Teardown a vector database provisioned for an agent fleet.

Provision a vector database for an agent fleet.

Get cost analytics

Delete a checkpoint.

List all checkpoints for a training job.

Promote a checkpoint to a final model path for serving.

Restore a specific checkpoint for a training job.

Manually trigger a checkpoint save for a running training job.

Configure provider credentials

Deep cost analysis of current GPU infrastructure: per-provider breakdown, utilization efficiency, waste identification, and optimization potential.

Generate actionable cost optimization recommendations: spot migration, GPU right-sizing, provider arbitrage, idle shutdown, and density packing.

Simulate cost optimization scenarios with ROI projections. Compare current vs optimized infrastructure costs.

Create a PostgreSQL database connection with auto-table creation. Returns a connection ID for subsequent operations.

Create a SQLite database connection with auto-table creation. Returns a connection ID for subsequent operations.

Create a Weaviate collection.

Delete a Weaviate collection.

Hybrid vector + BM25 search in a Weaviate collection.

Insert objects into a Weaviate collection.

List Weaviate collections.

Vector similarity search in a Weaviate collection.

Initialize a Weaviate connection.

Evaluate a single LLM output with a DeepEval metric (AnswerRelevancy, Faithfulness, Hallucination, etc.).

Generate a starter DeepEval test file.

List available DeepEval metrics for LLM evaluation.

Run a DeepEval test suite from a Python test file.

Show DVC diff between two revisions (e.g. training checkpoints). Shows added, modified, deleted files.

Push DVC-tracked data to the configured remote storage.

Atomic checkpoint staging: DVC add + push + git commit in one operation. Promotes a training checkpoint to versioned storage.

Get DVC repository status: tracked files, remotes, and changes since last commit.

Find the cheapest egress route between cloud providers/regions for model weights or dataset transfer. Supports multi-hop routing.

Optimize dataset or model staging across regions by finding the cheapest transfer plan. Integrates with the dataset stager for parallel uploads.

Approve a pending environment promotion.

Create a named deployment environment (dev/staging/prod) with config.

Show promotion/change history for an environment.

List deployment environments.

Promote a workload or config between environments.

Get information about a database connection including type, status, and configuration.

Generate comprehensive compliance report: consent stats, policy evaluations, data movements, violations. For GDPR/SOC2/HIPAA audits.

Evaluate OPA (Open Policy Agent) policies for data access. Checks region restrictions, classification rules, and compliance requirements.

Move data with full governance audit trail. Requires prior consent and OPA policy approval. Tracks integrity, encryption, and compliance.

Get data movement audit log. Filter by user, dataset, or time range.

Record a consent response (granted or denied) for a pending consent request.

Request user consent for data movement across cloud regions. GDPR/SOC2 compliant consent tracking with audit trail.

GPU NUMA topology report with intra-GPU XCD (Accelerated Compute Die) awareness. Models MI300X (8 XCDs, 192GB HBM3), MI300A (6 XCDs, 128GB), H200 (unified 141GB HBM3e), H100 (80GB). Reports PCIe locality (PIX/PXB/PHB/SYS), GPU-NIC pairing, SR-IOV VF status, and generates XCD-aware NCCL/AITER environment variables.

Send a message through NeMo Guardrails and return the safety-filtered response. Applies topical, jailbreak, PII, and factcheck rails.

Generate default Colang 2.x guardrails configuration files (topical, jailbreak, PII, factcheck rails).

Generate Kubernetes deployment manifest for NeMo Guardrails server (standalone or sidecar mode).

Test connection to NeMo Guardrails server.

Generate Helm charts from workload specifications.

Create a HuggingFace Inference Endpoint (paid GPU endpoint). Supports custom GPU types, regions, and scaling.

Delete a HuggingFace Inference Endpoint.

Run inference on a HuggingFace Inference Endpoint. Supports text generation, embeddings, and custom inputs.

Get detailed info about a specific HuggingFace Inference Endpoint: status, URL, scaling config, cost.

Compare all hardware options for a HuggingFace model. Returns side-by-side cost, performance, and compatibility analysis.

Get hardware recommendation with cost breakdown for any HuggingFace model. Returns optimal GPU type, estimated cost, and performance score.

Search and browse HuggingFace Hub datasets. Filter by author and search query.

List all active HuggingFace Inference Endpoints with status, URL, and cost.

Search and browse HuggingFace Hub models. Filter by author, task, library. Returns model ID, downloads, likes, and tags.

Get detailed model info: architecture, size, downloads, license, tags, pipeline_tag, and model card.

Auto-generate an optimized deployment template for any HuggingFace model. Analyzes model size, architecture, and quantization to select optimal hardware and generate ready-to-deploy configs.

Deploy model to HuggingFace Spaces

Get HuggingFace Space deployment status.

Run health checks and auto-failover for inference endpoints. If a primary endpoint is unhealthy and has a backup configured, traffic automatically shifts to the backup provider.

Semantic-aware inference routing. Analyzes query content across 6 signal dimensions (modality, complexity, domain, language, safety, keywords), applies NUMA-aware endpoint scoring, and selects the optimal inference endpoint. Uses DAG parallel execution for signal extraction.

Disaggregated Prefill/Decode routing (DistServe architecture). Splits LLM inference into compute-bound prefill phase (routed to FLOPS-optimized GPUs like H100 SXM) and memory-bound decode phase (routed to bandwidth-optimized GPUs like MI300X). Tracks KV cache handoffs between endpoint pairs.

Configure InferX serverless platform credentials.

Delete an InferX model deployment.

Deploy model to InferX serverless platform

List deployed InferX models

Get cost analysis for inference endpoints

Get InferX pricing quotes for a GPU type.

Check InferX endpoint status

Get InferX account usage statistics: requests, cost, GPU hours, latency.

Create Kubernetes cluster with GPU nodes for optimal multi-cloud deployment

Destroy a Kubernetes cluster

Configure Kubernetes GPU device plugin settings: time-slicing, MIG strategy, and resource naming.

Install NVIDIA GPU Operator on a Kubernetes cluster. Configures driver containers, device plugin, DCGM exporter, and GPU Feature Discovery.

Get information about a specific cluster

List Kubernetes clusters

Configure Multi-Instance GPU (MIG) partitioning on A100/H100 GPUs. Splits a single GPU into isolated instances for multi-tenant workloads.

Configure GPU time-slicing for Kubernetes. Allows multiple pods to share a single GPU with configurable oversubscription.

Create a Karpenter GPU NodePool.

Delete a Karpenter NodePool.

Show recent Karpenter scaling events.

List GPU nodes provisioned by Karpenter.

Install Karpenter on an EKS cluster for GPU node autoscaling.

Show Karpenter controller logs.

List Karpenter NodePools.

Show Karpenter resource usage and capacity.

Show Karpenter installation/controller status.

Generate a GPU-aware KServe InferenceService YAML manifest with NUMA pinning, resource limits derived from model size and VRAM, and topology hints.

List KServe InferenceServices in a Kubernetes namespace.

Get detailed status of a KServe InferenceService including readiness, traffic split, and URL.

Analyze KV cache efficiency across serving endpoints.

Create an SGLang model-serving pipeline via LangChain. Connects LangChain agents to SGLang inference endpoints.

Create a LangChain workflow.

Configure Langfuse credentials (public key, secret key, host URL).

List Langfuse datasets for evaluation and fine-tuning.

Export Langfuse traces as instruction/response pairs for LoRA fine-tuning. Filters by quality score.

Generate Kubernetes deployment manifest for self-hosted Langfuse.

Print OTEL environment variables for instrumenting LLM apps to send traces to Langfuse.

Get aggregated quality metrics from Langfuse scores for drift detection.

Create an evaluation score for a Langfuse trace (e.g. quality, accuracy, relevance).

List evaluation scores from Langfuse, optionally filtered by trace or score name.

Test Langfuse connectivity and list accessible projects.

Get a single Langfuse trace with all observations/spans.

List recent LLM traces from Langfuse.

Create a LangGraph stateful workflow with monitoring. Supports agent graphs, tool calling, and state persistence.

Create an evaluator-optimizer workflow in LangGraph. Generates outputs, evaluates quality, and iteratively improves.

Create an orchestrator-worker pattern workflow in LangGraph. The orchestrator delegates tasks to specialized worker agents.

Get the status and metrics of a LangGraph workflow execution.

Record a data/model lineage checkpoint.

Diff lineage between two artifacts or checkpoints.

Export lineage data (JSON/graph) for an artifact.

Render the lineage graph for an artifact.

Trace the lineage of a dataset or model artifact.

Scan local machine and network for available GPU devices. Returns total VRAM pool for local-first provisioning.

Hot-load a LoRA adapter onto a running vLLM endpoint. The adapter becomes immediately available as a model name for inference requests. Uses vLLM's fused_moe_lora kernel for 454% higher output tokens/sec on MoE models.

List LoRA adapters loaded on a running vLLM endpoint. Shows base models and hot-loaded fine-tuned adapters.

Hot-unload a LoRA adapter from a running vLLM endpoint. Frees GPU memory for other adapters.

Manage GPU instances (stop/start/terminate)

List cached manifests and versions for jobs.

Compare migration target providers by cost/compatibility.

Execute a previously-planned workload migration.

Plan a workload migration between providers with cost analysis.

Roll back a migration to the source provider.

Show status of running/completed migrations.

Load the active registry version of an adapter onto a vLLM server.

List LoRA adapters currently loaded on a vLLM server.

Hot-load a LoRA adapter onto a running vLLM server.

Synchronize an adapter from the registry across multiple vLLM replicas.

Hot-unload a LoRA adapter from a running vLLM server.

List MLflow experiments on the configured tracking server.

Log a Terradev training run to MLflow with auto-injected GPU type, provider, cost/hr, and duration as params.

Register a trained model in the MLflow model registry with Terradev provenance tags.

Deploy Mixture-of-Experts models with production-ready cluster templates. Auto-applies vLLM cost optimizations (KV cache offloading for up to 9x throughput, MTP speculative decoding for up to 2.8x speed, sleep mode for 18-200x faster restarts). Supports GLM-5, Qwen 3.5, Mistral Large 3, DeepSeek V4, Llama 5. Configures NVLink topology, tensor parallelism, FP8 quantization, vLLM/SGLang backends, and GPU-aware HPA autoscaling.

Chat with an Ollama model using the chat/completions API.

Generate text using an Ollama model (non-chat completions).

List models available on an Ollama server.

Get detailed information about an Ollama model (parameters, template, license).

List currently running Ollama models.

Pull a model to an Ollama server on a remote instance.

Find cheaper alternatives for running instances

Evict a model from GPU memory.

Test inference with a model via the orchestrator.

Load a model into GPU memory.

Register a model with the orchestrator.

Start the model orchestrator for multi-model GPU sharing with eviction policies.

Get orchestrator and model status including GPU memory utilization.

Generate Kubernetes deployment manifest for self-hosted Arize Phoenix server.

Generate OpenTelemetry environment variables for instrumenting serving pods with Phoenix tracing.

List Phoenix projects (trace namespaces).

Generate Python instrumentation snippet for adding Phoenix tracing to LLM applications.

List recent spans for a Phoenix project. Supports SpanQuery DSL filters like "span_kind == 'RETRIEVER'" or "status_code == 'ERROR'".

Test connection to Arize Phoenix server. Returns collector endpoint and project count.

View full execution tree for a specific trace ID. Shows span hierarchy, latencies, and token counts.

Pre-training validation: GPU availability, NCCL, RDMA, drivers across all nodes.

GPU-specific preflight validation: NVIDIA drivers, CUDA version, GPU count, NCCL, NVLink topology, NCU stall-signature profiling, and adversarial config verification (V1-V3).

Network-specific preflight validation: RDMA availability, InfiniBand status, inter-node bandwidth, latency matrix, firewall rules.

Run preflight validation before provisioning GPU instances.

Generate full preflight validation report with pass/warn/fail per check. Covers GPU drivers, CUDA, NCCL, RDMA, network, disk, and Docker.

GPU price intelligence with quantitative analytics. Computes delta (rate of change), gamma (acceleration), and annualized realized volatility on GPU spot/on-demand prices across 21+ providers. Identifies cheapest time windows and provider arbitrage opportunities.

Spot instance risk assessment per provider. Returns interruption probability, mean time to interruption, and recommended mitigation.

Get GPU price trend analysis with delta (rate of change), gamma (acceleration), and annualized volatility. Identifies cheapest time windows.

Provision GPU instances for optimal parallel efficiency

List all Qdrant vector collections with their point counts and configurations.

Count points (vectors) in a Qdrant collection.

Create a Qdrant vector collection. Auto-configures vector dimensions from embedding model name.

Get detailed info and stats for a Qdrant collection.

Generate Kubernetes StatefulSet manifest for self-hosted Qdrant vector database.

Test connection to Qdrant vector database. Returns cluster info and collection count.

Execute a SELECT query on a database connection. Returns query results as a list of dictionaries.

Get a GPU price quote across providers.

Generate a Ray Serve LLM disaggregated Prefill/Decode deployment. Splits inference into compute-bound prefill and memory-bound decode phases with KV cache transfer via NIXL.

List all running Ray jobs and tasks.

Compute optimal TP/DP/EP parallelism strategy for a given MoE model and GPU count. Returns recommended configuration with rationale.

Start a Ray cluster (head node or worker). For distributed ML training and inference.

Get Ray cluster status including node count, resources, memory, and running jobs.

Stop the Ray cluster on the current node.

Submit a job script to the Ray cluster for distributed execution.

Generate a Ray Serve LLM Wide-EP (Expert Parallel) deployment for MoE models. Returns Python script and config for distributed MoE serving with EPLB and DeepEP.

Explicit versioned rollback. Format: job@version (e.g., llama3@v3).

Run a declarative YAML workflow that chains multiple Terradev commands (provision → preflight → train → monitor → checkpoint). Returns step-by-step execution status with cost estimates and confirmation gates for expensive operations.

Print environment-style export lines for a provider. By default values are masked.

Retrieve a stored secret. By default the value is masked.

List stored provider and key names. Values are never shown.

Remove a provider or a single key from the secret store.

Run a shell command with secrets injected into the environment.

Store a secret in the Terradev secrets store. Values are encrypted at rest.

Import TERRADEV_* environment variables into the secret store for supported cloud providers.

Check which providers are fully configured and which keys are missing.

Get setup instructions for a provider

Complete SGLang optimization stack with workload-specific auto-tuning for 7 workload types: agentic chat, batch inference, low latency, MoE models, PD disaggregated, structured output, and RAG.

[DEPRECATED] Use sglang action='inference' instead. Test inference against a running SGLang endpoint.

[DEPRECATED] Use sglang action='start' instead. Start an SGLang inference server on a remote instance.

[DEPRECATED] Use sglang action='stop' instead. Stop an SGLang server on a remote instance.

Compress, chunk, checksum, and position datasets near compute. Supports local paths, S3/GCS URIs, HTTP URLs, and HuggingFace dataset names. Returns staging plan with agent recommendations.

View all instances and costs with state optimization

Launch distributed training on provisioned GPU nodes. Supports torchrun, deepspeed, accelerate, and megatron. Use from_provision='latest' to auto-resolve node IPs from your last provision command.

Detect straggler nodes in distributed training. Identifies GPUs with significantly lower utilization that slow the whole job.

Real-time GPU monitoring for training jobs. Shows utilization, memory, temperature, power, and cost.

Resume a training job from its latest checkpoint. Rebuilds config with topology revalidation.

Get complete training monitoring snapshot: GPU metrics (utilization, memory, temp, power), training metrics (loss, grad_norm, lr, throughput), straggler detection, and cost estimate.

List all training jobs and their state (created, running, completed, failed).

Stop a running training job. Kills training processes on all nodes in parallel.

Run an Unsloth local model server.

Start a coding agent backed by Unsloth's local model server.

Stop a running Unsloth server started with `unsloth run`.

Generate a complete training configuration from a declarative spec. Auto-detects framework, sets optimal parallelism, and configures distributed training.

Full distributed training launch with framework auto-detection, topology validation, and monitoring. Combines preflight + train + monitor in one operation.

Create an event trigger (schedule/webhook/metric).

Enable or disable a trigger.

Manually execute a trigger now.

Show trigger execution history.

List configured triggers.

CLI-native provisioning with manifest cache and drift detection. Use --fix-drift to detect and auto-fix drifted infrastructure.

Insert or update data in a database table. Performs upsert operation (insert or update on conflict).

Analyze a running vLLM server's workload and return specific optimization recommendations with before/after comparisons.

Automatically optimize vLLM configuration by analyzing workload patterns. Selects optimal settings for the 6 critical knobs based on live endpoint metrics or sample request files.

Benchmark a vLLM endpoint with concurrent requests. Returns throughput (req/s), success rate, and total latency.

Test inference against a running vLLM endpoint (completions or chat).

Get vLLM server info: loaded models, config, and health status.

Put a vLLM server to sleep. Level 1: offload to CPU (fast wake). Level 2: discard weights (minimal RAM).

Start a vLLM inference server on a remote instance via SSH/systemd. Supports Multi-LoRA, Sleep Mode, KV Offloading, Speculative Decoding.

Stop a vLLM server on a remote instance.

Wake a sleeping vLLM server. For Level 2 sleep, also reloads weights and resets prefix cache.

Create a custom W&B dashboard with GPU metrics, training loss, and cost panels.

Create a W&B report with custom sections, charts, and narrative text.

Auto-create standard Terradev alerts: GPU cost > budget, utilization < 50%, training loss spike, straggler detection.

Auto-create a Terradev-specific W&B dashboard with GPU utilization, cost tracking, training metrics, and infrastructure panels.

Auto-generate a Terradev infrastructure report: GPU costs, provider comparison, training efficiency, and recommendations.

Get comprehensive W&B monitoring overview: dashboards, reports, alerts, active runs.

List all Weights & Biases projects for the configured entity.

List runs in a W&B project with status, metrics summary, and config.

Get detailed info, metrics, and artifacts for a specific W&B run.

Set up custom W&B alerts for GPU metrics: cost thresholds, utilization drops, training anomalies.

Start the warm pool manager for intelligent model pre-warming. 5 strategies: traffic_based, time_based, priority_based, cost_optimized, latency_optimized.

Get warm pool status: hit rate, cold starts, memory saved, cost saved.