fuzzing-python

Files (1)

• SKILL.md

SKILL.md (first lines)

# Python Fuzzing Skill

## Setting up fuzzing for a Python project

Fuzz testing for Python projects are Atheris.
Atheris is a coverage-guided Python fuzzing engine.
It supports fuzzing of Python code, but also native extensions written for CPython.
Atheris is based off of libFuzzer.
When fuzzing native code,
Atheris can be used in combination with Address Sanitizer or Undefined Behavior Sanitizer to catch extra bugs.

You can install prebuilt versions of Atheris with pip:

```sh
pip3 install atheris
```

These wheels come with a built-in libFuzzer,
which is fine for fuzzing Python code.
If you plan to fuzz native extensions,
you may need to build from source to ensure the libFuzzer version in Atheris matches your Clang version.

## Using Atheris

### Example

```python
#!/usr/bin/python3

import atheris

with atheris.instrument_imports():
  import some_library
  import sys

def TestOneInput(data):
  some_library.parse(data)

atheris.Setup(sys.argv, TestOneInput)
atheris.Fuzz()
```

When fuzzing Python, Atheris will report a failure if the Python code under test throws an uncaught exception.

### Python coverage

Atheris collects Python coverage information by instrumenting bytecode.
There are 3 options for adding this instrumentation to the bytecode:

- You can instrument the libraries you import:

  ```python
  with atheris.instrument_imports():
    import foo
    from bar import baz
  ```

  This will cause instrumentation to be added to `foo` and `bar`, as well as
  any libraries they import.

- Or, you can instrument individual functions:

  ```python
  @atheris.instrument_func
  def my_function(foo, bar):
    print("instrumented")
  ```

- Or finally, you can instrument everything:

  ```python
  atheris.instrument_all()
  ```

  Put this right before `atheris.Setup()`. This will find every Python function
  currently loaded in the interpreter, and instrument it.
  This might take a while.

Atheris can additionally instrument regular expression checks, e.g. `re.search`.
To enable this feature, you will need to add:
`atheris.enabled_hooks.add("RegEx")`
To your script before your code calls `re.compile`.
Internally this will import the `re` module and instrument the necessary functions.
This is currently an experimental feature.

Similarly, Atheris can instrument str methods; currently only `str.startswith`
and `str.endswith` are supported. To enable this feature, add
`atheris.enabled_hooks.add("str")`. This is currently an experimental feature.

#### Why am I getting "No interesting inputs were found"?

You might see this error:

```
ERROR: no interesting inputs were found. Is the code instrumented for coverage? Exiting.
```

You'll get this error if the first 2 calls to `TestOneInput` didn't produce any
coverage events. Even if you have instrumented some Python code,
this can happen if the instrumentation isn't reached in those first 2 calls.
(For example, because you have a nontrivial `TestOneInput`). You can resolve
this by adding an `atheris.instrument_func` decorator to `TestOneInput`,
using `atheris.instrument_all()`, or moving your `TestOneInput` function into an
instrumented module.

### Visualizing Python code coverage

Examining which lines are executed is helpful for understanding the
effectiveness of your fuzzer. Atheris is compatible with
[`coverage.py`](https://coverage.readthedocs.io/): you can run your fuzzer using
the `coverage.py` module as you would for any other Python program. Here's an
example:

```bash
python3 -m coverage run your_fuzzer.py -atheris_runs=10000  # Times to run
python3 -m coverage html
(cd htmlcov && python3 -m http.server 8000)
```

From benchflow-ai/skillsbench, Apache-2.0.

More from benchflow-ai/skillsbench