assessing-vector-and-embedding-weaknesses

Files (5)

• LICENSE
• SKILL.md
• api-reference.md · references/
• standards.md · references/
• agent.py · scripts/

SKILL.md (first lines)

# Assessing Vector and Embedding Weaknesses

> **Authorized use only:** These tests interact with vector stores and embedding models in RAG systems you own or are authorized to assess. Embedding inversion and cross-tenant probing against systems you do not control may expose third-party data and is prohibited without authorization.

## Overview

Retrieval-Augmented Generation (RAG) systems convert documents into embedding vectors stored in a vector database (Pinecone, Qdrant, Weaviate, Chroma, pgvector, FAISS) and retrieve the nearest vectors to ground LLM responses. OWASP **LLM08:2025 Vector and Embedding Weaknesses** covers the security risks unique to this layer:

- **Embedding inversion** — embeddings are not one-way. A trained inversion model (or a black-box reconstruction attack) can recover substantial portions of the original text from its vector, leaking source documents (maps to MITRE ATLAS **AML.T0024.001 Invert ML Model**).
- **Membership inference** — querying whether a specific record contributed to the corpus (AML.T0024.000).
- **Cross-tenant / multi-tenant leakage** — when one namespace/collection is shared or filter isolation is missing, a tenant retrieves another tenant's chunks.
- **Knowledge-base poisoning** — an attacker who can write to the corpus inserts crafted chunks that dominate retrieval (high cosine similarity to expected queries) and carry indirect prompt-injection payloads.
- **Retrieval manipulation** — adversarial documents tuned to be retrieved for many unrelated queries ("retrieval hijacking").

The parent technique is **AML.T0024 — Exfiltration via ML Inference API**: an attacker uses legitimate inference/query access to exfiltrate data (source text via inversion, membership, or model extraction). This skill provides a repeatable assessment of all five weakness classes.

## When to Use

- During a security assessment of any RAG / vector-search application (OWASP LLM08 coverage).
- When a vector store is multi-tenant and you must prove namespace/metadata isolation.
- When the corpus accepts user-supplied or third-party documents (poisoning surface).
- When the embedding endpoint is externally reachable (inversion/membership surface).
- When validating retrieval-filtering controls before go-live.

## Prerequisites

- Authorization and scope covering the target embedding endpoint and vector store.
- Python 3.10+.
- Read (and, for poisoning tests, write) access to a test collection — never the production corpus.

```bash
# Vector DB clients + embeddings + similarity tooling
python -m pip install numpy scikit-learn sentence-transformers
python -m pip install qdrant-client chromadb pinecone-client weaviate-client
# (optional) text-embedding inversion research baseline
python -m pip install vec2text
```

## Objectives

- Measure embedding-inversion exposure on the target embedding model.
- Run a membership-inference probe against the corpus.
- Test multi-tenant isolation (namespace, metadata filter, RBAC) for cross-tenant leakage.
- Inject benign poisoned chunks into a *test* collection and measure retrieval dominance.
- Detect indirect prompt-injection content surviving in retrieved chunks.
- Recommend controls: tenant-scoped filters, content validation, embedding-access limits.

## MITRE ATT&CK Mapping

| ID | Tactic | Official Technique Name | Role in this skill |
|----|--------|-------------------------|--------------------|
| AML.T0024 | ATLAS: Exfiltration | Exfiltration via ML Inference API | Using query/embedding access to exfiltrate source data |
| AML.T0024.000 | ATLAS: Exfiltration | Infer Training Data Membership | Membership-inference probe against the corpus |
| AML.T0024.001 | ATLAS: Exfiltration | Invert ML Model | Embedding-inversion reconstruction of source text |
| AML.T0020 | ATLAS: Resource Development | Poison Training Data | Knowledge-base poisoning of the corpus |
| AML.T0051.001 | ATLAS: Initial Access | LLM Prompt Injection: Indirect | Injection payloads embedded in retrieved chunks |

## Workflow

### Step 1: Inventory the RAG pipeline

Document the embedding model + dimensions, the vector store and its tenancy model, the chunking strategy, retrieval `top_k` and similarity metric (cosine/dot/L2), and any metadata filters applied at query time.

```python
# Example: inspect a Qdrant collection
from qdrant_client import QdrantClient
client = QdrantClient(url="http://localhost:6333")
info = client.get_collection("docs")
print(info.config.params.vectors)   # size + distance metric
print(client.count("docs"))         # corpus size
```

### Step 2: Test embedding-inversion exposure

Embeddings of similar text are close; an attacker with the embedding endpoint can iteratively reconstruct text whose embedding matches a target vector. Measure how much a nearest-neighbour-in-embedding-space recovers, using cosine similarity between candidate reconstructions and the target.

```python
import numpy as np
from sentence_transformers import SentenceTransformer
from sklearn.metrics.pairwise import cosine_similarity

model = SentenceTransformer("all-MiniLM-L6-v2")
secret = "Patient John Doe, MRN 553120, diagnosed with hypertension."
target_vec = model.encode([secret])

# Attacker has only target_vec and the embedding endpoint. Hill-climb candidate text.
candidates = [
    "Patient name and medical record number with a diagnosis.",
    "John Doe medical record hypertension diagnosis",
    "Patient John Doe MRN diagnosed hypertension",
]
cand_vecs = model.encode(candidates)
sims = cosine_similarity(target_vec, cand_vecs)[0]
for c, s in sorted(zip(candidates, sims), key=lambda x: -x[1]):
    print(f"{s:.3f}  {c}")
# High similarity for a near-verbatim guess => inversion risk is real for this model.
```

For a research-grade reconstruction baseline, `vec2text` can be used against compatible embedding models to demonstrate full-text recovery.

### Step 3: Membership inference

Determine whether a specific document is in the corpus by measuring the top-1 retrieval similarity for an exact-quote query: in-corpus items return a markedly higher max similarity than out-of-corpus controls.

```python
def membership_score(client, collection, embed, text):
    vec = embed([text])[0].tolist()
    hits = client.search(collection_name=collection, query_vector=vec, limit=1)
    return hits[0].score if hits else 0.0

in_corpus = membership_score(client, "docs", model.encode, "<exact quote from a known chunk>")
control  = membership_score(client, "docs", model.encode, "An unrelated random sentence.")
print(f"in-corpus={in_corpus:.3f}  control={control:.3f}  delta={in_corpus-control:.3f}")
# A large positive delta indicates the item is in the corpus (membership leak).
```

### Step 4: Test multi-tenant isolation

Confirm that tenant B cannot retrieve tenant A's chunks. Issue tenant-B-authenticated queries that *should* be filtered, and verify no tenant-A `tenant_id` appears in results.

From adriannoes/awesome-agentic-ai, Apache-2.0.

More from adriannoes/awesome-agentic-ai