CategoryQuality

A benchmark win is not the finish line

A
Profile again: The benchmark is not the finish line

TL;DR: Once a benchmark shows a win, put the optimized code back into the profiling harness. Compare the before and after memory and CPU profiles to see whether the larger execution path benefits too. A good benchmark table feels great. The before number is slower, the after number is faster, allocations drop, and the ratio looks impressive. After hours of staring at profiler stacks and benchmark output, that table feels like the finish line. Unfortunately, the table only describes the isolated...

Turn messy production code into a useful benchmark

T

TL;DR: Useful benchmarks are controlled experiments. Copy the relevant production code, trim away unrelated work, choose realistic parameters, measure one responsibility, and use short runs for direction before spending time on full runs. Most benchmark examples look cleaner than the code we work with. Suspiciously cleaner. They compare string concatenation with StringBuilder. They call a static method. They pass one value in and return one value out. Those examples are useful for learning...

Read profiles without chasing every red bar

R

TL;DR: Profilers show cost, not priority. Start with memory, then CPU, use filters to zoom into the code path you own, and let domain context decide which hot spots deserve a benchmark. The first profile is usually disappointing. You attach the profiler, run the profiling harness, take a snapshot, and get a giant list of allocations, call stacks, framework methods, runtime methods, transport code, serializer code, and your own code hiding somewhere in the middle. The tool did its job. It showed...

Build a profiling harness before you benchmark

B

TL;DR: Before writing a benchmark, build a small profiling harness that makes the code path visible. Run it in Release mode, keep unrelated work out, add clear profiler snapshot points, and collect both memory and CPU evidence. Production code is a terrible place to start a performance investigation. There is too much happening at once. Production code is not bad; it is alive. It has real configuration and input/output, plus logging, retries, dependency injection, serialization, network calls...

Stop guessing: the performance loop for production code

S

TL;DR: A benchmark can tell you whether code got faster. It cannot tell you whether the code mattered. For that, use a loop: profile with a profiling harness, improve a hot path, benchmark and compare, profile again, then ship and observe production. The first benchmark I wrote looked deceptively easy. I needed a class, a few attributes, and a method. Then I could run BenchmarkDotNet and get a table. It looked a lot like unit testing, which made me dangerously confident. That confidence did not...

To test, or not to Test? Part 3 – Make it easier to recover from a defect

T

I wrote code without tests that ran in production without defects, and I wrote buggy code with TDD (Test Driven Development). Time to look back at 35 years of coding and when tests help, and when there is something better. And especially, what these better things are.

In this post, we look at what we can do to recover well even if a defect finds its way into production.

To test, or not to Test? Part 2 – Make it hard to make mistakes

T

I wrote code without tests that ran in production without defects, and I wrote buggy code with TDD (Test Driven Development). Time to look back at 35 years of coding and when tests help, and when there is something better. And especially, what these better things are. In this part: What would you do if you weren’t allowed to write automated tests? Take a moment and think about what you would do to still deliver quality software if you weren’t allowed to write automated tests. I...

Our journey to F#: The effect of F# on our (unit) tests

O

On our journey from C# to F#, we were at a point where we changed our (unit) testing strategy. Our approach just didn’t feel right anymore. We struggled to do TDD because we didn’t get the tests to fail first. We wrote tests that didn’t increase our confidence and started thinking of them as waste.

This post is part of the F# Advent Calendar 2021 
Thanks to Sergey Tihon for organising the Advent Calendar.

Recent Posts