Software Practice, Testing and Debugging

Version control, tests, reading and writing code others can use, how work grows with data, profiling, common bugs and a review checklist.

Version control with Git

Git records every change to your project as a commit, so you can see what changed, why, and by whom, return to any earlier state, and work on an idea without risking the working version. It is the single most useful habit in this guide, and it works for one person on one laptop as well as for a team.

TaskCommandNotes
Start a projectgit initCreates the hidden .git folder
See what changedgit status, git diffRun before every commit
Record a changegit add file.py then git commit -m "Fix baud rate rounding"Commit small, working steps; the message says why
Historygit log --onelineOne line per commit
Try an idea safelygit switch -c experimentA branch; merge it back with git merge or throw it away
Undo uncommitted edits to a filegit restore file.pyDestroys the edits: check git diff first
Go back to an old versiongit switch --detach abc1234Look around without changing anything
Mark a releasegit tag v1.2So that a measurement can name the exact code that produced it
Sharegit remote add origin URL, git pushGitHub, GitLab or a plain server: a backup and a way to collaborate
  • Commit often, with clear messages: an imperative summary in 50 characters, then, if needed, a blank line and the reason. In 40 commits you can still find which one changed the filter.
  • Ignore generated files with .gitignore (compiled output, virtual environments, large data, passwords): .venv/, __pycache__/, *.o, secrets.json.
  • Keep data and code separate; large or private data belong outside the repository (or in Git LFS), with a README on how to get them.
  • Never commit secrets (passwords, API keys, Wi-Fi credentials). If you do, change them: removing the file later does not remove it from history.
  • Tag the code behind every important result (a test report, a released firmware) so that you can reproduce it.

Readable code

Code is read far more often than it is written, mostly by you, six months later. Clear code is the cheapest way to avoid bugs.

# Hard to read
def f(a, b, c):
    return a * (b - 32) / 1.8 + c

# Easier: names with units, one job, a docstring, no magic numbers
FAHRENHEIT_OFFSET = 32.0
FAHRENHEIT_PER_CELSIUS = 1.8

def fahrenheit_to_celsius(temp_f: float) -> float:
    """Convert degrees Fahrenheit to degrees Celsius."""
    return (temp_f - FAHRENHEIT_OFFSET) / FAHRENHEIT_PER_CELSIUS
  • Name things for what they are, with units: speed_mm_s, timeout_ms, is_homed. A good name makes most comments unnecessary.
  • Functions do one thing and fit on a screen; if you need “and” to describe one, split it.
  • No magic numbers: give constants names and put them in one place.
  • Comments explain why, not what: “the sensor needs 80 ms to settle” is useful, “increment i” is noise. Keep them true when the code changes.
  • Type hints and docstrings (Python), const and fixed-width types (C): the compiler or a checker (mypy) finds mistakes for you.
  • Avoid global variables; pass values in and return results, so a function can be tested alone.
  • Be consistent with a style guide (PEP 8 for Python) and let a formatter (black, ruff, clang-format) apply it.
  • Delete dead code: it is in the history if you need it.

Testing

A test is code that runs your code on a known input and checks the answer. Tests catch the mistake in minutes instead of months, and they let you change code without fear: when something breaks, the test tells you where.

# test_numeric.py  (run with: pytest)
import math, pytest
from numeric import newton, rms, parse_line

def test_newton_sqrt_two():
    assert newton(lambda x: x*x - 2, lambda x: 2*x, 2.0) == pytest.approx(math.sqrt(2), rel=1e-12)

def test_rms_of_sine_is_amplitude_over_root_two():
    import numpy as np
    t = np.linspace(0, 1, 1000, endpoint=False)
    assert rms(5 * np.sin(2 * np.pi * 50 * t)) == pytest.approx(5 / math.sqrt(2), rel=1e-9)

def test_parse_rejects_bad_checksum():
    assert parse_line("$TEMP,23.41,C*00") is None          # a corrupted line must not become data

def test_empty_input_raises():
    with pytest.raises(ValueError):
        rms([])

Worked example: what to test

  • Known answers: √2 = 1.414213562373…; the RMS of a sine is A ÷ √2; a hand-solved circuit; a unit conversion (212 °F is 100 °C).
  • Compare floats with a tolerance: pytest.approx (or math.isclose), never == (see Numbers and Floating Point).
  • Edge cases: empty input, one element, the largest and smallest values, zero (division!), negative numbers, NaN, and a missing file.
  • Bad input: a corrupted line, a timeout, a value out of range: the code should reject it cleanly, not crash or guess.
  • Regression tests: when you find a bug, first write a test that fails because of it, then fix it; the test then guards against it for good.
  • Properties: things that must always hold (a sorted list stays the same length; filtering never increases noise; converting there and back returns the start); the hypothesis library generates hundreds of cases.
Kind of testWhat it checksRun
UnitOne function in isolationOn every change, in seconds
IntegrationParts working together (parser and database)On every change or nightly
Hardware in the loopReal device with a test rig that stimulates itBefore release; automated where possible
AcceptanceThe requirement, as the user sees itBefore delivery
Static analysisLint and type checks, compiler warnings, no running neededOn every change
  • Separate the logic from the hardware so that most code can be tested on a PC with fake inputs (see Embedded Systems: Debugging and Testing).
  • Run the tests automatically (a hook, or a service such as GitHub Actions) so that a failure is seen at once.
  • Coverage (pytest --cov) shows which lines the tests never ran: the bugs hide there. 100 % coverage is not proof of correctness, only of effort.
  • Test the tests: change the code on purpose and confirm that a test fails.

Debugging method

  • 1. Reproduce it reliably, with the smallest input that fails. Write down what you did.
  • 2. Read the error message, all of it, and the line it points at: the answer is often there. A stack trace is read from the bottom (where it happened) up.
  • 3. Form a hypothesis and test it, rather than changing things at random. Ask: what do I expect this value to be here, and what is it?
  • 4. Look at the data: print it, plot it, or step to it in the debugger (breakpoint() in Python, gdb or the IDE in C). Print the types and the sizes too.
  • 5. Bisect: halve the problem: does the bug exist before this step? git bisect finds the commit that introduced a regression automatically.
  • 6. Check the assumptions: units, byte order, indexes from 0 or 1, a stale file, the wrong library version, the wrong port.
  • 7. Explain it to someone (or a rubber duck): saying it aloud often reveals the mistake.
  • 8. Fix the cause, add a test, and look for the same mistake elsewhere.

Common bugs

BugWhat happensFix
Off-by-oneA loop runs one time too many or too few; an index goes past the endRemember arrays start at 0; check the first and last pass; prefer iterating over items
Integer overflowA counter or product wraps and gives a wrong or negative numberUse a wider type; convert before multiplying (see Numbers and Floating Point)
Integer division1 / 2 is 0 in CConvert an operand to float first
Comparing floats with ==Equal values test as differentCompare with a tolerance
Uninitialized variableWorks on one run, fails on anotherGive every variable a value when it is declared
Wrong unitsResults are off by a constant factor (25.4, 1 000, 60)Put units in names; test against a hand calculation
Byte order and signMulti-byte values are scrambled or negative numbers come out hugeCheck the specification (see Bits, Bytes and Data Formats)
Mutating shared dataA function changes data that another part still uses (aliasing in Python)Copy it; pass values; keep shared state small
Race conditionsA value changes between two reads (an interrupt or thread)Disable interrupts briefly or lock shared data
Resource leaksFiles, ports or memory are not released; a program slows and then fails after daysUse with (Python) or RAII (C++); close what you open
Silent failureAn error is caught and ignored, so wrong data flow onCatch specific exceptions; log and handle them
Hidden dependence on order or stateA notebook cell or function works only after another has runPass inputs explicitly; restart and run everything
Copy-and-paste editsOne copy is fixed, the other forgottenMake it a function

How work grows with data

How work grows with the number of items051015200100200300400number of items nstepsnn log₂ nn²2ⁿ (clipped)At n = 20: 20 steps, 86, 400, and 1 048 576 steps. At n = 1 000 an n² method needs a million steps; 2ⁿ is impossible.
The number of steps against the number of items n for four kinds of algorithm.

The way the work grows with the amount of data (written O(n), “order n”) decides whether a program that is fine on a test file survives the real data. Doubling n doubles an O(n) job, quadruples an O(n²) job, and squares the time of an O(2n) job.

Worked example: a million items

  • Finding one value in a million: scanning a list takes up to 1 000 000 comparisons (O(n)); a binary search of a sorted list takes ⌈log2 1 000 000⌉ = 20; a dictionary or set lookup takes about one (O(1)).
  • Sorting a million items by a good algorithm takes about n log2 n = 2.0e+7 steps, 0.02 s at a billion simple steps per second. A naive method that compares every pair takes n² = 1e+12 steps = 1 000 s (16.7 minutes): the same code that took 1.0 ms on 1 000 items.
  • The fix is usually data structure, not speed: a set for “have I seen this?”, a dict for lookups by key, sorting once and searching many times, and avoiding a loop inside a loop over the same data.
  • Do not optimize what is not slow: a loop over 100 items does not matter at any complexity.
  • Vectorize numeric work (NumPy) instead of looping in Python (see Python for Engineers).
  • Watch memory too: a list of a million floats takes about 32 MB in pure Python and 8 MB as a NumPy array.

Profiling and making code faster

Do not guess where the time goes: measure it. A profiler reports how much time each function takes.

python -m cProfile -s cumtime analyse.py | head -20        # which functions take the time

import timeit
timeit.timeit("sorted(data)", globals=globals(), number=100)     # time one statement

# in a notebook:  %timeit f(x)      and      %prun analyse()

Worked example: where to spend effort

  • Amdahl’s law: if 80 % of the run time is in one part and you make that part 10 times faster, the whole program is only 1 ÷ ((1 − 0.8) + 0.8 ÷ 10) = 3.57 times faster, and no improvement of that part can beat 1 ÷ (1 − 0.8) = 5.0 times. Find the biggest part first.
  • Is it worth it? A script that runs 200 times at 12 s each, speeded up to 0.4 s, saves 200 × 11.6 s = 0.6 hours in total: less than the 3 hours it took to write the faster version? Weigh the savings against the effort, and the risk of new bugs.
  • Order of attack: a better algorithm, then vectorize, then cache results you recompute, then compile the hot loop (Numba, Cython, C), then more hardware.
  • Measure again after each change and keep the tests running: a fast wrong answer is worse than a slow right one.

Error handling and logging

  • Decide what to do on every failure: retry, use a default, skip and flag, or stop. “Ignore it” is a decision too, so make it deliberately and write it down.
  • Fail early and loudly on conditions that cannot be recovered (a missing config, a wrong data version); validate inputs at the boundary.
  • Catch specific exceptions, add context when you re-raise (raise RuntimeError("reading n3") from e), and never write a bare except: pass.
  • Use a logger, not print: levels (debug, info, warning, error), time stamps and a file. logging.basicConfig(filename="run.log", level=logging.INFO).
  • Log what you need to reproduce a failure: the input, the version, the settings and the time; not secrets.
  • For machines, add a safe state: what the outputs do when the program crashes or the link drops (see Embedded Systems: Architecture).

Review, documentation and sharing

Review questionWhy
Does it do what the requirement says, including the edge cases?Most bugs are missing cases, not typos
Are units, byte order and sign handled and written down?The expensive mistakes
What happens on bad input, a timeout, a full disk or a lost connection?Real systems fail
Can I understand it without the author? Are names and comments clear?The next reader is you
Are there tests, and did they run?Confidence to change it
Are there secrets, hard-coded paths or magic numbers?Portability and security
Is it safe if it fails (outputs, limits, stops)?Code that moves things can hurt
  • Write a README: what it does, how to install and run it, the inputs and outputs with units, an example, the version and the license.
  • Document the why (decisions, assumptions, the datasheet page) beside the code, and keep a changelog for released tools.
  • Pin dependencies (requirements.txt, a lock file) and say which versions of Python and the libraries you tested.
  • Pick a license for code you share (MIT, Apache, GPL) and check the licenses of what you use.
  • Review each other’s code: a second pair of eyes finds what you cannot see. For safety-related code, review is a requirement, not a favor.
  • Using an AI coding assistant: it can write plausible code quickly and make plausible mistakes just as quickly. Read every line, run it, test it against known answers, and never paste secrets or confidential data into a service you do not control. You remain responsible for the result.

Safety, security and good habits

  • Code that controls machines needs hardware protection that works when the software does not: limit switches, thermal fuses, emergency stops, watchdogs.
  • Test with the actuators disconnected first, then with limits and low power, then on the real machine with a hand on the stop.
  • Treat all input as untrusted: files, network messages and serial lines can be malformed or malicious; check lengths and ranges, and never build commands or SQL from raw text.
  • Keep secrets out of code (environment variables or a protected file), and keep software updated.
  • Back up your work (Git on a remote, plus a copy elsewhere) and test the restore.
  • Safety-critical software (guarding, medical, vehicles, aviation) follows standards (IEC 61508, ISO 13849, ISO 26262, DO-178C) with certified tools and processes: a script is not enough (see System Integration).
  • Take breaks and stop when tired; most careless bugs are written late in the day.

Related

FAQ

Do I really need version control for a small script?

Yes: it takes a minute to set up and saves you the day you break something and cannot remember what you changed. It is also the best backup and the only reliable record of which code produced which result.

How many tests are enough?

Enough that you are not afraid to change the code: tests for the core calculations with known answers, for every bug you have fixed, and for the edge and failure cases. Start with a few; add one for every bug.

My program is slow: should I rewrite it in C?

Almost never first. Profile it, fix the algorithm, vectorize with NumPy, and only then compile the one hot loop (Numba, Cython, or a C extension). Rewriting everything costs more than it gains.

How do I read someone else’s code?

Start from the entry point and the tests; run it in a debugger and watch the data move; read the README and the commit history; draw the call structure. Do not try to read it like a book.

References

  • Hunt, A. and Thomas, D., The Pragmatic Programmer; McConnell, S., Code Complete; Martin, R. C., Clean Code.
  • Chacon, S. and Straub, B., Pro Git (free online); the pytest, Hypothesis and Python logging documentation.
  • Wilson, G. et al., “Best practices for scientific computing”, PLOS Biology, 2014; Amdahl, G., “Validity of the single processor approach to achieving large scale computing capabilities”, 1967.

Last reviewed: October 2026. Timings and counts are computed from the stated rates: measure your own code.