BenchmarkDotNet

BenchmarkDotNet helps you to transform methods into benchmarks, track their performance, and share reproducible measurement experiments. It's no harder than writing unit tests! Under the hood, it performs a lot of magic that guarantees reliable and precise results thanks to the perfolizer and pragmastat statistical engine. 

How to use it?

Let's say you have a solution with this structure

MyApp
├- MyApp.UI
├- MyApp.Services
├-Myapp.Domain

1- add new console project to your solution (e.g. MyBenchmarks)
2- install nuget package in MyBenchmarks => dotnet add package BenchmarkDotNet 
3- reference MyApp.Services  to MyBenchmarks

4- Create a benchmark class

using BenchmarkDotNet.Attributes;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.DependencyInjection;


[MemoryDiagnoser]
public class InboxServiceBenchmarks
{
    private InboxService _service;

    [Params(1, 2, 3)]
    public long UserId;

    [GlobalSetup]
    public void Setup()
    {
        var opts = new DbContextOptionsBuilder<AppDbContext>().UseSqlServer("connection string").Options;
        var db = new AppDbContext(opts);
        _service = new InboxService(db);
    }

    [Benchmark]
    public Task<List<InboxModel>> LoadInbox()
    {
        return _service.LoadInbox(UserId, CancellationToken.None);
    }
}

5- add more benchmark class as well

6-  change program.cs

using BenchmarkDotNet.Running;

BenchmarkRunner.Run<InboxServiceBenchmarks>();

//#for multiple benchmark class
//BenchmarkRunner.Run(new[] { typeof(InboxServiceBenchmarks), typeof(Class2Benchmarks) });
//or
//BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args);

7- start your console project MyBenchmarks in release mode (in debug you’ll get error)
      BenchmarkDotNet requires Release builds because Debug optimizations are disabled and results become unreliable.

8- Sample result
          Method     | Mean       | Error      | StdDev      | Gen0  | Allocated

          LoadInbox | 12.34 ms  | 0.25 ms | 0.30 ms     |  5.2     | 32 KB
       
Mean (MOST IMPORTANT) => performance
 Average execution time per operation (On average, this method takes ~12.34 milliseconds per run.)
✔ This is your main performance number
✔ Lower = faster

Error => confidence
Estimated uncertainty of the mean (The real mean is likely within ±0.25 ms)
✔ Smaller = more reliable benchmark
✔ High error = noisy environment

StdDev => consistency

How much results vary between runs (Each execution fluctuates around the mean by ~0.30 ms)

✔ Low = stable performance
❌  High = inconsistent method or noisy system

Gen0 / Gen1 / Gen2 =>  memory pressure

Shows the amount of garbage collection activity normalized per benchmark operation. These numbers help identify allocation-heavy code and potential GC overhead.

✔ Lower is generally better
✔ 0 Gen0 often =  very few allocations
❌ High values usually indicate many short-lived allocations

Allocated =>  memory pressure

Total memory allocated per operation (Each execution of this method allocates ~32 KB)

✔ Lower is better

BenchmarkDotNet Cheat Sheet

1. you can have Multiple Methods

   public class StringBenchmarks
   {
          [Benchmark]
          public string Plus() => "A" + "B";
   
          [Benchmark]
          public string Interpolation() => $"{'A'}{'B'}";
   
          [Benchmark]
          public async Task<int> AsyncMethod()
          {
                 await Task.Delay(10);
                 return 1;
          }
   }

2. Memory Allocation

   [MemoryDiagnoser]
   public class MyBenchmarks
   {
             [Benchmark]
             public byte[] Allocate() => new byte[1024];
   }

Output includes:

• Mean
• Error
• StdDev
• Gen0 / Gen1 / Gen2
• Allocated

3. Parameters

   public class ListBenchmarks
   {
       [Params(10, 100, 1000)]
       public int Count;
   
       [Benchmark]
       public List<int> Create()   => Enumerable.Range(0, Count).ToList();
   }

4. Global Setup / Cleanup

   public class MyBenchmarks
   {
       private List<int> _data;
   
       [GlobalSetup]
       public void Setup()=>_data = Enumerable.Range(1, 100000).ToList();
   
       [GlobalCleanup]
       public void Cleanup()=>      _data.Clear();
   
       [Benchmark]
       public int Sum() => _data.Sum();
   }

5. Per-Iteration Setup : Runs before/after every benchmark iteration.

   [IterationSetup]
   public void IterationSetup()  {}
   
   [IterationCleanup]
   public void IterationCleanup()  {}

6. Baseline Comparison : Shows ratio relative to baseline.

   [Benchmark(Baseline = true)]
   public void OldMethod()  {}
   
   [Benchmark]
   public void NewMethod()  {}

7. Return Values : BenchmarkDotNet prevents dead-code elimination.

   [Benchmark]
   public int Sum()
   {
       return Enumerable.Range(1, 100).Sum();
   }

8. Different Jobs (.NET Versions)

   [SimpleJob]
   [MemoryDiagnoser]
   public class MyBenchmarks
   {
   
   }

OR:

[SimpleJob(RuntimeMoniker.Net80)]
[SimpleJob(RuntimeMoniker.Net90)]

9. Categories & Filtering : Group benchmarks and run only specific categories.

   [BenchmarkCategory("EF")]
   [Benchmark]
   public Task<List<InboxModel>> LoadInbox()
   {
       return _service.LoadInbox(UserId, CancellationToken.None);
   }

Run a specific category: dotnet run --filter * --category EF

Useful when you have dozens of benchmarks in the same project.

10. Export Results: BenchmarkDotNet can export reports in multiple formats.

    using BenchmarkDotNet.Exporters;
    
    [MarkdownExporter]
    [HtmlExporter]
    [CsvExporter]
    public class MyBenchmarks
    {
    }

Common outputs:

• Markdown report
• HTML report
• CSV file for Excel analysis

11. Additional Diagnosers: Besides memory allocation, BenchmarkDotNet provides other diagnosers.

    [MemoryDiagnoser]    // Allocations and GC activity
    [ThreadingDiagnoser] // Threading and lock statistics
    [ExceptionDiagnoser] // Exception counts during execution
    public class MyBenchmarks
    {
    }

12. Job Configuration: Control how BenchmarkDotNet executes benchmarks.

    using BenchmarkDotNet.Jobs;
    
    [ShortRunJob]
    public class MyBenchmarks
    {
    }
    
    
    // This allows comparing the same code across .NET versions
    [SimpleJob(RuntimeMoniker.Net80)]
    [SimpleJob(RuntimeMoniker.Net90)]
    public class MyBenchmarks
    {
    }

Common jobs:

13. Benchmark Categories in Reports: Organize large benchmark suites.

    [BenchmarkCategory("EF Core")]
    [Benchmark]
    public void EfMethod() { }
    
    
    [BenchmarkCategory("Dapper")]
    [Benchmark]
    public void DapperMethod() { }

This makes reports easier to analyze when comparing multiple technologies.

14. Common Report Columns: Additional columns can help analyze results.

    using BenchmarkDotNet.Columns;
    
    [MinColumn]
    [MaxColumn]
    [RankColumn]
    public class MyBenchmarks
    {
    }

Useful columns:

Most Common Attributes

For EF Core vs Dapper comparisons, the most useful combination is usually:

[MemoryDiagnoser]
[RankColumn]
[Orderer(BenchmarkDotNet.Order.SummaryOrderPolicy.FastestToSlowest)]

because it shows execution time, memory usage, and relative ranking in one report.

Common Benchmarking Mistakes
❌ Running in Debug
❌ Benchmarking against production DBs
❌ Logging inside benchmark methods
❌ Allocating setup data inside [Benchmark]
❌ Running benchmarks while other heavy workloads are active