How do you profile your research software to find performance bottlenecks?
Description
Profiling tells you where your code is actually spending its time and memory, rather than where you assume it is. For research software, this matters because a single slow loop or an unnecessary data copy can dominate runtime, and intuition about where the bottleneck lives is frequently wrong. Profiling before optimising ensures you spend your limited effort on the part of the code that will actually make a measurable difference.
Considerations
- Guessing where the bottleneck is, rather than measuring it, is one of the most common ways researchers waste time on optimisation that doesn’t help.
- Different profilers measure different things — CPU time, wall-clock time, memory allocation, or GPU usage — so choose one that matches the resource you suspect is the constraint.
- Profiling overhead can distort results, especially for fine-grained or sampling profilers on short-running code, so treat absolute numbers with some caution and focus on relative comparisons.
- Representative input data matters: profiling on a tiny test case can hide bottlenecks that only appear at the data sizes you actually run in production.
- The right profiler depends on your language and environment, and tool support varies a lot between ecosystems such as Python, C/C++, Fortran, MATLAB, and OpenMP-parallelised code.
Solutions
- Start with a coarse-grained profiler to identify which functions or modules consume the most time, before drilling into line-level detail.
- For Python, built-in options such as cProfile give a quick first pass; for compiled languages, tools such as perf or vendor profilers are common starting points.
- Once you know which function is slow, use a line-level or statistical profiler to see exactly which statements inside it cost the most.
- Re-profile after each significant change to confirm the bottleneck has actually moved or shrunk, rather than assuming a fix worked.
- The SIG-RPC profiler database lists language-specific profiling tools and guidance for C/C++, Fortran, MATLAB, OpenMP, and Python, and is a practical starting point if you are unsure which tool fits your codebase.
- If you are new to profiling entirely, the SIG-RPC getting started guide walks through the basic workflow from a fresh perspective.
How do you optimise your research software once you know where the bottleneck is?
Description
Optimisation is the process of changing code so it does the same job using fewer resources, typically less time, memory, or energy. It should follow profiling, not precede it: optimising code that isn’t actually the bottleneck wastes effort and adds complexity for no benefit. For research software, the goal is usually “reasonable” performance — exhausting the easy, low-risk wins — rather than squeezing out every possible cycle at the cost of code clarity or correctness.
Considerations
- Optimisation often trades code readability or generality for speed, so weigh whether the gain is worth the added maintenance burden, especially in code other researchers will need to understand later.
- Many performance problems have language- or library-specific fixes — for example, vectorising operations in Python or MATLAB, or improving memory access patterns in C/C++ and Fortran — so generic advice only goes so far.
- Algorithmic changes (a better data structure, a lower-complexity algorithm) usually yield far larger gains than low-level micro-optimisations, and are worth considering before tuning small details.
- Parallelism (multi-threading, multi-processing, or GPU offload) can deliver large speedups but introduces its own correctness risks, such as race conditions, and is a separate consideration from sequential optimisation.
- Any optimisation must preserve correctness: re-run your tests after each change, since a faster but wrong result is worse than a slow but correct one.
- If you use AI tools to help generate or refactor code for performance, the resulting code needs the same scrutiny, testing, and profiling as any other change — speed claims from generated code should not be trusted without verification.
Solutions
- Confirm the bottleneck with a profiler before changing anything, and re-profile afterwards to verify the change had the intended effect.
- Look for algorithmic improvements first: a change from an O(n²) approach to an O(n log n) one will usually outperform any amount of low-level tuning.
- Apply language-specific techniques next, such as avoiding unnecessary copies, using vectorised operations instead of explicit loops, or reducing redundant I/O.
- Consider parallel approaches (multi-threading, MPI, or GPU acceleration) only once sequential improvements are exhausted and the workload genuinely suits parallel execution.
- The SIG-RPC optimisations database provides language-specific optimisation guidance for C/C++, Fortran, MATLAB, OpenMP, Python, and R, with concrete techniques rather than abstract advice.
- The SIG-RPC case studies show how real research codebases identified and resolved performance issues, which is useful for seeing the profiling-to-optimisation workflow applied end to end.
Further Reading
- SIG-RPC getting started guide — A practical introduction to profiling workflow from the Reasonable Performance Computing Special Interest Group, aimed at researchers who are new to performance work and want a clear first step.
- SIG-RPC profiler database — A curated, language-specific listing of profiling tools covering C/C++, Fortran, MATLAB, OpenMP, and Python, useful for quickly identifying which profiler fits your stack.
- SIG-RPC optimisations database — Concrete, language-specific optimisation techniques for C/C++, Fortran, MATLAB, OpenMP, Python, and R, going beyond profiling into actual fixes.
- SIG-RPC case studies — Worked examples of real research software performance investigations, showing how profiling findings translate into optimisation decisions in practice.
- Python documentation: The Python Profilers — Official reference for
cProfileandprofile, the standard built-in entry point for profiling Python research code without installing extra tooling.
AI Disclosure
This work was produced with the assistance of Claude Sonnet 4.6, under the strict editorial control and factual verification of the human author.
Related pages
tasks
How to decide which programming languages, tools and infrastructures to use?
tasks
How to measure and improve environmental sustainability of software?
tasks
Guidance on keeping research software functioning, understandable, and usable over time — covering ongoing maintenance practices, dependency management, documentation, versioning, and the management of technical debt.