AGENTS.md · blacklanternsecurity/bbot

Sections

1. Core Principles
2. Tooling
3. Dev Environment Setup
4. AI Use Disclosure
5. Architecture Overview
6. Writing a Module

First lines (shown under the repo's AGPL-3.0 license)

# BBOT Developer Guide

## Core Principles

### Modularity Principle
When writing a BBOT module, make sure all module-specific code lives in the module itself. Don't hard-code module-specific things in core or in helpers.

### DRY Principle
Don't Repeat Yourself -- and interpret this broadly. If two pieces of code aren't identical but follow a similar enough pattern that they could be generalized, they should be. Extract shared logic into a common abstraction rather than duplicating the pattern. Usually this means creating a shared helper, or a shared module template in `bbot/modules/templates`. When you notice structural similarity, unify it.

### Engineering Principle
Every system that is implemented must be implemented properly. No hacks, no hardcoding, no shortcuts. If we implement one of something, we build a proper system for it. It's okay to take a step back from the current task, in order to do things right. This relates directly to the Modularity Principle above.

### Testing Principle
BBOT has extremely thorough tests, including **one or more individual tests for each module, with no exceptions**. This is critical to maintaining stability in a recursive tool, which by its nature flirts with race conditions and infinite loops. If you add a module, you write a test. If you change a module, you make sure its test still passes.

---

## Tooling

- **Package manager**: [uv](https://docs.astral.sh/uv/)
- **Linter/formatter**: [ruff](https://docs.astral.sh/ruff/) (pinned to 0.15.10)
- **Test framework**: [pytest](https://docs.pytest.org/) with pytest-asyncio
- **Python**: 3.10 - 3.14

---

## Dev Environment Setup

```bash
# 1. Fork and clone
git clone git@github.com:<you>/bbot.git
cd bbot

# 2. Switch to dev branch, then create a feature branch
git checkout dev
git checkout -b my-feature

# 3. Install uv (if you haven't already)
curl -LsSf https://astral.sh/uv/install.sh | sh

# 4. Install all dependencies (including dev)
uv sync --group dev

# 5. Install pre-commit hooks (ruff, file checks, etc.)
uv run pre-commit install

# 6. Activate the virtualenv
source .venv/bin/activate

# 7. Verify
bbot --help
```

### Running Tests

```bash
# Run the full suite
./bbot/test/run_tests.sh

# Run specific module tests
./bbot/test/run_tests.sh robots,sslcert

# Run a single test file directly
pytest bbot/test/test_step_2/module_tests/test_module_robots.py -x -vv
```

### Linting

```bash
ruff check          # lint
ruff format          # auto-format
ruff format --check  # verify formatting without changes
```

### Git Workflow

- `stable` - production releases
- `dev` - active development, **almost all PRs should target this branch**
- Feature branches should be created from `dev`

---

## AI Use Disclosure

Use of AI is not prohibited -- and in many cases, encouraged. However, when reviewing a PR, it is helpful for the reviewer to know the extent to which AI was used, and which model.
Please add a small section at the bottom of the PR with the header: `### AI Use Disclosure`, followed by the following information:

* Extent of the AI use. For example, was this fully autonomous by the AI, or was it a collaborative back-and-forth, or did the user just use the AI to review their work, etc.
* Model Used

This should only apply to external contributors, not members of the blacklanternsecurity organization.

---

## Architecture Overview

### How a Scan Works

BBOT is an async, recursive OSINT tool. A scan starts with **seed events** (targets) and passes them through a pipeline of **modules**. Each module watches for specific event types, processes them, and may emit new events, which feed back into the pipeline. This continues until no module has anything left to do.

```
                     Seeds (targets)
                          |
                          v
                   +--------------+
                   | ScanIngress  |  dedup, blacklist, scope check
                   +--------------+
                          |
                          v
                   +--------------+
                   |  Intercept   |  dns, cloud tagging
                   |   Modules    |  (modify/tag events before distribution)
                   +--------------+
                          |
                          v
                   +--------------+
                   |  ScanEgress  |  scope filtering, graph management
                   +--------------+
                          |
                          v
              +-----------+-----------+
              |           |           |
           Module A    Module B    Module C   ...
              |           |           |
              +-----------+-----------+
                          |
                          v
                   Output Modules (json, csv, neo4j, ...)
```

### Events

Events are the currency of BBOT. Every piece of data -- a hostname, IP, URL, open port, finding -- is an event. Events have:

- **type**: `DNS_NAME`, `IP_ADDRESS`, `URL`, `OPEN_TCP_PORT`, `HTTP_RESPONSE`, `FINDING`, `EMAIL_ADDRESS`, etc.
- **data**: the actual data (a string, dict, etc.)
- **parent**: the event that led to this one (forming a discovery chain)
- **scope_distance**: how many hops from the original target (0 = in-scope)
- **tags**: metadata like `in-scope`, `affiliate`, `cloud-azure`, `open-port`, etc.
- **module**: which module discovered it

### Scope Distance

Scope distance tracks how far an event is from the original target:
- `0` = explicitly in-scope (matches target or discovered in-scope)
- `1` = one hop away (e.g. a hostname found in an SSL cert of an in-scope host)
- `2+` = further away

The scan's `scope.search_distance` (default 0) controls how far modules are allowed to look. A module's `scope_distance_modifier` adjusts this per-module.

### Helpers

BBOT has a helper for almost everything. **Please use them.** They're accessible via `self.helpers` inside any module.

Key helpers:

| Helper | What it does |
|--------|-------------|
| `self.helpers.request(url)` | Make an HTTP request (with retries, SSL handling, etc.) |