benchee-profiling

Files (1)

• SKILL.md

SKILL.md (first lines)

# Benchee Profiling



## RULES — Follow these with no exceptions

**1.** **Profile first** — run `:fprof` or `:eprof` and verify the output explicitly names the expected slow call site; re-run with a larger workload if ambiguous
**2.** **Write a comparative benchmark** — implement at least 2 alternative approaches using Benchee, ensuring implementations do the same thing; benchmark in `MIX_ENV=prod` for realistic results
**3.** **Use multiple inputs** — test with small, medium, and large realistic data sizes to catch size-dependent behavior
**4.** **Warm up before measuring** — use `warmup: 2` and `time: 10`; repeat 3–5 times to rule out variance; if results are within noise (< 5% difference), run 3 additional times
**5.** **Validate improvement** — confirm the faster approach wins across all input sizes
**6.** **Save baseline and check regressions** — write results to `bench/baseline.json`; raise an error if performance degrades more than 10% (compare against the previous 3 baselines before raising to rule out noise)
**7.** **Separate I/O benchmarks** — never benchmark network or disk I/O in the same run as compute benchmarks


## Setup

```elixir
# mix.exs
defp deps do
  [
    {:benchee, "~> 1.3", only: :dev}
  ]
end
```


## Basic Benchmark with Full Configuration

Use this pattern as the starting point for any new benchmark — it covers timing, memory, multiple inputs, and formatted output in one call.

```elixir
# bench/list_benchmark.exs
Benchee.run(
  %{
    "Enum.sort" => fn list -> Enum.sort(list) end,
    "Enum.sort_by" => fn list -> Enum.sort_by(list, & &1) end
  },
  time: 10,              # Run each scenario for 10 seconds
  warmup: 2,             # Warm up for 2 seconds
  memory_time: 2,        # Measure memory usage
  reduction_time: 2,     # Measure reductions
  inputs: %{
    "small list"  => Enum.to_list(1..100),
    "medium list" => Enum.to_list(1..10_000),
    "large list"  => Enum.to_list(1..100_000)
  },
  formatters: [
    {Benchee.Formatters.Console, comparison: true},
    {Benchee.Formatters.HTML, file: "output/benchmark.html"}
  ]
)
```

```bash
# Run benchmark
MIX_ENV=prod mix run bench/list_benchmark.exs
```


## Comparing Real-World Implementations

Use this pattern when you have multiple concrete implementations of the same operation and need to confirm which is fastest across realistic data. Unlike the basic example above, this section demonstrates benchmarking non-trivial logic defined in a module.

```elixir
defmodule StringOperations do
  def concat_loop(strings) do
    Enum.reduce(strings, "", fn s, acc -> acc <> s end)
  end

  def concat_join(strings) do
    Enum.join(strings)
  end

  def concat_comprehension(strings) do
    for s <- strings, into: "", do: s
  end
end

strings = for i <- 1..1000, do: "string_#{i}"

Benchee.run(%{
  "reduce"        => fn -> StringOperations.concat_loop(strings) end,
  "join"          => fn -> StringOperations.concat_join(strings) end,
  "comprehension" => fn -> StringOperations.concat_comprehension(strings) end
})
```


## Profiling with :fprof

```elixir
:fprof.trace(:start, file: 'trace.trace')
MyApp.SlowFunction.run()
:fprof.trace(:stop)

:fprof.profile(file: 'trace.trace')
:fprof.analyse(dest: 'analysis.txt')
```


## Profiling with :eprof

```elixir
:eprof.start()

:eprof.start_profiling([self()])
MyApp.SlowFunction.run()
:eprof.stop_profiling()

:eprof.analyze()
:eprof.stop()
```


## Benchmark Suite Organization

```
bench/

From igmarin/elixir-phoenix-skills, MIT.

More from igmarin/elixir-phoenix-skills