TL;DR: Once a benchmark shows a win, put the optimized code back into the profiling harness. Compare the before and after memory and CPU profiles to see whether the larger execution path benefits too.
A good benchmark table feels great.
The before number is slower, the after number is faster, allocations drop, and the ratio looks impressive. After hours of staring at profiler stacks and benchmark output, that table feels like the finish line.
Unfortunately, the table only describes the isolated benchmark.
Before trusting the result, take it back to the larger execution path.
A benchmark compares a controlled slice. The real workload may tell a different story.
Removing noise made the comparison possible, but production code has to run with all of those surrounding costs.
Put the improved code back into the profiling harness and profile the larger path again.
Put the change back where it has to live
The copy-and-trim benchmark helped compare two versions of pipeline invocation. It did that by removing unrelated dependencies and measuring one responsibility. That was the right tool for the inner loop: improve, benchmark, learn, improve again.
Once the benchmark shows a meaningful improvement, the question changes. We no longer ask only “is this operation faster in isolation?” We ask “does the surrounding system show the benefit?”
That means moving the optimized code back into the profiling harness from earlier in the series. Use the same workload, same profiler workflow, and same snapshot points where possible. The comparison is only useful if the setup stays stable.
This second pass is where compounding effects become visible. A tiny allocation removed from one pipeline invocation may disappear thousands of times across a message run. Or it may vanish in the noise because another subsystem dominates the path.
Either answer is useful.
Memory should tell the same story
Start with memory again. In the original pipeline profiles, behavior-chain and delegate allocations showed up in the relevant call stacks. After the optimization, those allocations should be gone or reduced in the same view.
The before-and-after screenshots matter because they tell a story the benchmark cannot. The benchmark says the isolated pipeline invocation allocates less. The profile says those allocations were also removed from the larger publish run.
Use the same trick for the receive path. If the benchmark affected shared pipeline infrastructure, both publish and receive may benefit. If only one path changes, that is useful information too. It tells you where the optimization actually applies.
CPU should change shape too
Memory is only half the picture. The CPU profile should show whether the framework now spends less time on pipeline infrastructure.
Flame graphs make that visible without forcing us to start with exact percentages. Do not read the method names first. Read the shape first.
In the talk, I described it as less red and more orange: less infrastructure code dominating the picture, more business code becoming visible.
The benchmark improved one operation. The second profile shows whether that improvement compounds across the larger path.
In the second profile, the improved number should come with a visible change in shape.
In the pipeline investigation, the CPU overhead around publish and receive operations dropped from the relevant profiler views. That was the confirmation we wanted.
The benchmark improvement was not trapped inside the benchmark. It showed up in the harness.
Expect the system-level gain to be smaller
A five-times faster benchmark does not mean the whole application becomes five times faster.
That is normal.
The larger path still has serialization, transport code, persistence, transactions, logging, user handlers, database calls, network calls, and runtime overhead. If the optimized operation is only part of the total execution time, the total improvement will be smaller than the benchmark ratio.
That does not make the change disappointing. If a shared path runs millions of times per day, a few percent at the system level can be a good result. Less allocation pressure can also improve tail latency and reduce garbage collection work even when average timing changes look modest.
The benchmark shows that the change worked in isolation. The second profile reveals how much it matters inside the real call stack.
After the profile found the cost and the benchmark compared alternatives, production still has to confirm that the improvement mattered.
Production gets the final vote
The profiling harness is still not production. It is closer than the benchmark, but still controlled. The final answer comes after shipping and observing real workloads.
Watch the signals that should move: throughput, latency, allocation rate, garbage collection behavior, CPU usage, instance count, queue processing rate, and cloud spend. The exact signals depend on the system. A library team may look for benchmark and customer-reported improvements. An application team may look at dashboards, traces, and cost reports.
Production can invalidate assumptions. Maybe the code path was not as hot as the profiling harness made it look. Maybe a database call dominates everything. Maybe the workload changed. That is not failure if the team learns from it.
Write the lesson down. Add it to the pull request, a decision record, or a note near the code. Include what was measured, what changed, why the added complexity was accepted, and what production signal should improve.
Evidence also tells you when to stop
The performance loop can continue forever if you let it. Profile, improve, benchmark, profile again, ship, observe. Then another allocation appears. Another hot spot looks interesting. Another idea shows up.
Stop when the gain no longer justifies the complexity or the time. Stop when the hot path has moved somewhere else. Stop when production data says the next bottleneck is outside the code you are tuning.
Stopping is an engineering decision too.
Performance work competes with features, reliability, support, documentation, and everything else the team needs to do. Each pass through the loop produces evidence that makes the choice to continue or stop less arbitrary.
Once the team trusts the improvement, the next question is how to protect it from regression.
Further reading
- Pipeline before/after profiling screenshots
- 10x faster execution with compiled expression trees
- 5x faster pipeline execution by removing closure allocations
Common questions
What if the benchmark improved but the profiling harness profile barely changed?
The optimized operation may be too small in the larger path, or the profiling harness may not exercise the same scenario. Re-check the assumptions before adding complexity. The benchmark can be correct and still not matter enough.
When is the optimization good enough?
When the gain justifies the complexity, tests protect the behavior, and production has a signal you expect to improve. If you cannot name the signal, you may not be ready to ship the optimization.
Should I always profile again after benchmarking?
For important changes, yes. The second profile is what connects the benchmark back to the system. Without it, you know the isolated operation improved, but you do not know whether the surrounding path noticed.
Performance loop status
- [x] Understand the loop
- [x] Build profiling harness
- [x] Profile
- [x] Improve / Benchmark
- [x] Profile again
- [ ] Ship and observe
