TL;DR: Profilers show cost, not priority. Start with memory, then CPU, use filters to zoom into the code path you own, and let domain context decide which hot spots deserve a benchmark.
The first profile is usually disappointing.
You attach the profiler, run the profiling harness, take a snapshot, and get a giant list of allocations, call stacks, framework methods, runtime methods, transport code, serializer code, and your own code hiding somewhere in the middle.
The tool did its job. It showed what happened. Now you have to decide what matters.
That distinction is easy to miss. A profiler can show where memory was allocated or where CPU time was spent. It cannot know your product priorities, ownership boundaries, customer usage, or risk tolerance. That context comes from the team.
Start with memory because allocations compound
I usually start with memory profiles. In .NET, allocations are often the first place to look because allocation pressure compounds. Allocate too much on a hot path and the garbage collector eventually has to clean it up. That cleanup costs CPU, can interrupt useful work, and may force the system to use more capacity than it should.
Allocation fixes are also often easier to reason about than CPU fixes. Removing a temporary array, avoiding a closure, reusing a buffer, or moving setup work out of the hot path is usually less risky than rewriting an algorithm. Not always, but often enough that memory is a good first pass.
In the NServiceBus pipeline investigation, the memory profile showed a lot of activity. Some of it came from the transport. Some came from serializers and stream handling. Some came from the pipeline itself. The important part was not the largest number on the screen. The important part was finding allocation patterns connected to pipeline invocation.
Zoom in before you draw conclusions
Most profilers let you filter by namespace, type, or method. Use that aggressively. If you are investigating an order-processing controller, filter toward that controller and the code below it. If you are investigating a message pipeline, filter toward the pipeline namespace and pipeline invocation stack.
Filtering does not make the rest of the system disappear. It helps you ask a narrower question.
In the pipeline profile, zooming in showed behavior-chain allocations, delegate allocations, and display-class allocations. Display classes are a useful clue because they often point to closures. A closure is not automatically bad. But a closure allocated on a hot path deserves attention.
Resist the urge to fix things immediately. At this stage, the useful move is to turn “the process allocates a lot” into “this specific path allocates these specific objects when this specific operation runs.”
Large numbers are not automatically priorities
Large numbers are seductive because they look like priorities.
Sometimes they are, but they can just as easily pull the investigation toward code that is irrelevant or outside the team’s control.
The profiling harness used MSMQ because it was convenient and local. The memory profile showed allocations connected to MSMQ. That did not make MSMQ the best target.
MSMQ was not the reason for the investigation. It was outside the useful ownership boundary, and its user base was shrinking compared with transports such as RabbitMQ, Azure Service Bus, or Amazon SQS. Spending time there would help fewer users and would not improve the core pipeline.
The pipeline itself was different. Any improvement there could benefit every transport, because every message crosses the pipeline boundary. That made the smaller, more focused allocation pattern more interesting than some larger numbers elsewhere in the process.
A small improvement on a shared path can compound across every transport.
- Is this code under our control?
- Does fixing it help many users or only one narrow path?
- Do we understand it well enough to change it safely?
- Can a smaller fix on a hotter shared path compound across the system?
Domain knowledge turns profiler output into priorities.
Use CPU profiles to see shape, not just percentages
After memory, take a CPU profile. CPU profiles can be harder to read because the cost may be spread across runtime helpers, virtual calls, asynchronous state machines, locks, delegates, and user code. A flame graph helps because it shows shape.
In a flame graph, wider bars represent more total time. A parent bar includes the work below it. You do not need to understand every method name to get value. If a wide band of infrastructure code surrounds a small amount of domain work, the picture already tells a story.
Do not read the method names first. Read the shape first.
That was the useful signal in the publish pipeline profile. The framework infrastructure took a large share of the visible CPU before customer work had much room to appear. That does not prove a specific fix, but it does justify a deeper look.
Flame graphs are not magic. They are a way to see relationships quickly. Once a shape looks suspicious, use the profiler’s call tree or hot spots view to confirm the path and quantify the cost.
Hot spots turn suspicion into evidence
Most profilers also have a hot spots view. It ranks methods or call paths by cost and gives percentages. In the pipeline investigation, the publish path showed a meaningful share of CPU time in behavior-chain invocation. The receive path was less dramatic, but still showed infrastructure overhead worth reducing.
The profiler can point at expensive code, but it cannot decide whether that code is a sensible target.
It knows nothing about your organizational boundaries.
The team supplies that context.
The exact number matters less than the decision it supports. We now have a candidate: pipeline invocation creates memory pressure and CPU overhead on a shared path. That candidate is specific enough to benchmark.
The profile is the beginning, not the verdict
A profile is evidence rather than a verdict. The harness shapes what appears in it through the chosen transport, workload, runtime configuration, machine, and profiler. Treat those results as a starting point.
The next step is to isolate the hot path and compare a change. BenchmarkDotNet becomes useful at that point. Benchmarks are not better than profiles. They answer the next question: did this specific change improve this specific operation under controlled conditions?
Use the profile to choose a target and the benchmark to compare the change.
Keeping those jobs separate prevents a lot of wasted tuning.
Further reading
Common questions
Should I always fix the largest allocation first?
No. Fix the largest relevant allocation that you can safely change and that benefits the path you care about. A smaller allocation on a hotter shared path can be a better target than a larger allocation in code you do not own.
What if I do not understand the hot spot?
Stop and build knowledge. Talk to the owning team, read the code, or choose a better-understood target first. Performance changes made without understanding can turn a slow bug into a fast bug.
Do I need flame graphs?
No, but they help. A call tree or hot spots view can be enough. Flame graphs are useful because they show relationships quickly, especially when infrastructure code wraps the work you care about.
Performance loop status
- [x] Understand the loop
- [x] Build profiling harness
- [x] Profile
- [ ] Improve
- [ ] Benchmark
- [ ] Profile again
- [ ] Ship and observe
