Data Logging

A data logger records measurements over time so that they can be analyzed later: to troubleshoot a machine, prove that a process stays in tolerance, monitor energy, test a prototype or train a model. The hard part is rarely storing numbers. It is deciding what to record, how fast and how accurately, and making sure that the numbers are correct, in order and correctly timed, long after you have forgotten how the logger was set up. This guide takes the logging chain from the sensor to the report in nine pages: planning, sensors and converters, sampling and aliasing, time stamps and clocks, storage and formats, logger hardware, logging strategies, telemetry and dashboards, and data quality. Every worked number is computed so that you can check your own system against it. The code and file formats for reading data back are in Data, Files and Parsing.

Start here: what do you need?

I need to…	Go to
decide what to log and how long to keep it	Planning a Logging System
choose a sensor, a converter, or find out why readings are noisy or wrong	Sensors, Signal Conditioning and ADCs
pick a sample rate, avoid aliasing, or design a filter	Sampling, Aliasing and Filtering
get trustworthy time stamps, or synchronize several loggers	Time Stamps and Clocks
estimate storage, pick a format, or stop an SD card wearing out	Storage, Formats and Data Rates
choose a logger: a commercial unit, a microcontroller, a Pi or a PLC	Logger Hardware: Dedicated, Embedded and Industrial
capture rare events, or store less without losing what matters	Logging Strategies: Triggers, Deadbands and Aggregation
send data to the cloud, build a dashboard or an alert	Telemetry, Databases and Dashboards
check, analyze and report data honestly	Data Quality, Analysis and Reporting

A suggested order

Start with Planning a Logging System, because every later choice follows from it. Then work along the chain: Sensors, Signal Conditioning and ADCs, Sampling, Aliasing and Filtering and Time Stamps and Clocks decide the quality of each number; Storage, Formats and Data Rates, Logger Hardware: Dedicated, Embedded and Industrial and Telemetry, Databases and Dashboards decide whether it survives and gets seen; Logging Strategies: Triggers, Deadbands and Aggregation and Data Quality, Analysis and Reporting decide whether it is useful.

1. Planning a Logging System: The questions to answer first: what to measure, how fast, how accurately, for how long, and what happens when data is lost.
2. Sensors, Signal Conditioning and ADCs: The measurement chain from sensor to number: thermocouples, RTDs, 4–20 mA, bridges, converters, noise, grounding and an uncertainty budget.
3. Sampling, Aliasing and Filtering: Nyquist, how aliases appear, anti-alias filter design, oversampling and decimation, and choosing the sample rate and record length.
4. Time Stamps and Clocks: UTC and ISO 8601, clock drift in ppm, NTP, PTP and GPS, jitter, stamping at the source and synchronizing several loggers.
5. Storage, Formats and Data Rates: Data volume per day, text against binary, compression, SD card endurance, databases, file rotation and retention.
6. Logger Hardware: Dedicated, Embedded and Industrial: Commercial loggers, microcontrollers with SD, Raspberry Pi, PLC trends and polling devices over a network, with power, enclosure and runtime.
7. Logging Strategies: Triggers, Deadbands and Aggregation: Periodic, on-change, pre-trigger ring buffers, min/mean/max aggregation and burst capture, and what each saves and hides.
8. Telemetry, Databases and Dashboards: MQTT, store-and-forward, time-series databases, Grafana dashboards, alerts, data costs and securing a remote logger.
9. Data Quality, Analysis and Reporting: Validation, gaps, calibration, uncertainty, control charts, summary statistics, honest plots and a troubleshooting table.

The logging pipeline

Stage	What can go wrong	Where to look
Sensor and installation	Wrong range, slow response, poor contact, heat conduction	Sensors, Signal Conditioning and ADCs
Conditioning and ADC	Noise, offset, aliasing, a poor reference	Sensors, Signal Conditioning and ADCs, Sampling, Aliasing and Filtering
Logger and clock	Drifting time, missed samples, power loss	Time Stamps and Clocks, Logger Hardware: Dedicated, Embedded and Industrial
Storage	Full disk, corrupted files, a worn card	Storage, Formats and Data Rates
Network and database	Lost messages, duplicates, no buffering	Telemetry, Databases and Dashboards
Analysis and reporting	Hidden gaps, wrong units, misleading plots	Data Quality, Analysis and Reporting

Quick reference: rules and formulas

Topic	Rule or formula	Page
Nyquist	fs > 2 fmax; in practice 5 to 10 × the highest frequency of interest	Sampling, Aliasing and Filtering
Alias frequency	| f − k fs | with k the nearest whole number	Sampling, Aliasing and Filtering
RC filter cut-off	fc = 1 ÷ (2π R C)	Sampling, Aliasing and Filtering
Oversampling gain	+½ log2 N bits for N averaged samples	Sensors, Signal Conditioning and ADCs
Uncertainty (independent)	u = √(u1² + u2² + …)	Sensors, Signal Conditioning and ADCs
Clock drift	seconds per day = ppm × 0.0864	Time Stamps and Clocks
Data per day	channels × bytes × rate × 86 400	Storage, Formats and Data Rates
Control limits	center line ± 3 σ of the subgroup means	Data Quality, Analysis and Reporting

Related guides and tools

• Data, Files and Parsing and Numerical Methods: reading, parsing, fitting and filtering the logs.
• Sensors and Field Devices: the sensors, their outputs and wiring.
• Embedded Systems: ADCs, timers, storage and low power on a microcontroller logger.
• Industrial Communication: reading drives, meters and PLCs over Modbus and Ethernet.
• Analog Scaling Tool: converting 4–20 mA and 0–10 V signals to engineering units.
• Machine Learning: using logged data for prediction and anomaly detection.

Safety and responsibility

• A logger is not a safety system. If a reading protects people or equipment (a gas, pressure or temperature limit), the protection must be a rated device that acts without the logger.
• Mains and process wiring is dangerous: use rated, isolated inputs and a competent person for anything above extra-low voltage.
• Data can be personal or confidential: energy, occupancy and location logs reveal habits; secure remote access and know who may see the data (see Telemetry, Databases and Dashboards).
• Regulated records (food safety, pharmaceutical, environmental) have rules on calibration, audit trails and retention: follow them from the start (see Data Quality, Analysis and Reporting).

FAQ

How fast should I sample?

At least twice the highest frequency in the signal, and in practice 5 to 10 times the frequency of interest, with an anti-alias filter in front of the converter. See Sampling, Aliasing and Filtering.

Should I log raw counts or scaled values?

Log values in engineering units for use, and keep the scaling and calibration in the metadata so that the raw counts can be recovered. See Planning a Logging System.

Which time zone should the log use?

UTC, in ISO 8601, with the source of the clock recorded. Convert to local time only for display. See Time Stamps and Clocks.

Where did the old Data Logging sections go?

All of them are in this guide with much more depth: sampling and aliasing, resolution, time stamps, storage size and formats, strategies, hardware and data quality each have their own page.

References

• JCGM 100:2008 (GUM), Guide to the expression of uncertainty in measurement; Bell, S., A Beginner’s Guide to Uncertainty of Measurement, NPL.
• Oppenheim, A. V. and Schafer, R. W., Discrete-Time Signal Processing; Smith, S. W., The Scientist and Engineer’s Guide to Digital Signal Processing.
• Montgomery, D. C., Introduction to Statistical Quality Control; ISO 8601 (date and time format); RFC 5905 (NTP).

Last reviewed: October 2026. Specifications and prices change: check the datasheet of the part you use.