Bits, Bytes and Data Formats

Binary, hexadecimal and decimal

A bit is 0 or 1; a byte is 8 bits (values 0 to 255, or 0x00 to 0xFF). Hexadecimal writes four bits per digit (0–9, A–F), so one byte is two hex digits and a 32-bit word is eight: 1011 0010 is 0xB2. Hardware registers, protocols, memory dumps and file formats are described this way because hex converts to bits digit by digit, which decimal does not.

Decimal	Hex	Binary	Octal
10	0xA	0000 1010	12
178	0xB2	1011 0010	262
255	0xFF	1111 1111	377
1000	0x3E8	1111101000	1750
4660	0x1234	1001000110100	11064

• Place values: binary digits are worth 128, 64, 32, 16, 8, 4, 2, 1. 1011 0010 = 128 + 32 + 16 + 2 = 178. Hex digits are worth 16ⁿ: 0xB2 = 11 × 16 + 2 = 178.
• Prefixes: 0x for hex and 0b for binary in C (C23), C++14 and Python; a leading 0 means octal in C (so 010 is 8, a classic trap).
• Powers of two to know: 2⁸ = 256, 2¹⁰ = 1 024, 2¹⁶ = 65 536, 2²⁰ = 1 048 576, 2³² = 4 294 967 296.
• Kilo, mega, giga: a kilobyte is 1 000 bytes (kB) and a kibibyte is 1 024 (KiB); memory sizes use the binary meaning, disk and data rates the decimal one.

Bit operations and masks

Operation	C / Python	Effect	Common use
AND	a & b	1 only where both bits are 1	Test a bit; clear bits (with an inverted mask)
OR	a | b	1 where either bit is 1	Set bits
XOR	a ^ b	1 where the bits differ	Toggle bits; simple checksums; swapping
NOT	~a	Flips every bit	Build an inverted mask (in Python it gives a negative number: mask the result)
Shift left	a << n	Moves bits up n places (multiplies by 2ⁿ)	Build a mask: 1 << n; pack fields
Shift right	a >> n	Moves bits down n places (divides by 2ⁿ, rounding down)	Extract a field

#define BIT(n)  (1u << (n))

reg |=  BIT(2);            /* set bit 2    */
reg &= ~BIT(4);            /* clear bit 4  */
reg ^=  BIT(7);            /* toggle bit 7 */
if (reg & BIT(1)) { }      /* test bit 1   */

/* extract a 3-bit field that starts at bit 4 */
uint8_t field = (reg >> 4) & 0x07;

For masks and register fields on a real chip see the Bitwise and Register Field Calculator.

Register fields and packing data

Hardware registers and compact protocols put several small values in one word. The recipe is always: mask to isolate bits, shift to move them, OR to combine.

def pack_word(reading, status):
    return ((status & 0xF) << 12) | (reading & 0xFFF)

def unpack_word(word):
    return word & 0xFFF, (word >> 12) & 0xF       # (reading, status)

assert unpack_word(pack_word(0xA5C, 0x9)) == (0xA5C, 0x9)

Two’s complement and sign extension

Signed integers are stored in two’s complement: the top bit has a negative weight. To negate a number invert all bits and add 1. This makes addition and subtraction work identically for signed and unsigned values, which is why every processor uses it. An n-bit signed number runs from −2ⁿ⁻¹ to 2ⁿ⁻¹ − 1: for 16 bits -32 768 to 32 767.

def sign_extend(value, bits):
    sign = 1 << (bits - 1)
    return (value & (sign - 1)) - (value & sign)     # no branch needed

sign_extend(0xFFE, 12)        # -2
sign_extend(0x7FF, 12)        # 2047

• Casting to a wider signed type extends the sign automatically; extracting a short field from a word does not, which is why you do it by hand.
• Offset binary (used by many ADCs and DACs) stores 0 as the middle value (2048 for 12 bits); subtract the offset to get a signed number.
• The most negative number has no positive partner: −(−128) overflows in 8 bits.

Byte order (endianness)

A multi-byte number is stored as bytes in some order. Big-endian puts the most significant byte first (the order of network protocols and of Modbus registers); little-endian puts the least significant byte first (x86, ARM Cortex-M in its usual setting, and most microcontrollers).

import struct
raw = bytes([0x42, 0xF6, 0xE9, 0x79])
struct.unpack(">f", raw)[0]        # big-endian float: 123.456...
struct.unpack("<f", raw)[0]        # the same bytes read little-endian: a different number
int.from_bytes(b"\x12\x34", "big")      # 4660
int.from_bytes(b"\x12\x34", "little")   # 13330

Other codes: BCD, Gray, parity and checksums

• BCD (binary-coded decimal) stores each decimal digit in four bits: 59 is 0101 1001 = 0x59. Real-time-clock chips and some displays use it; convert with (v >> 4) * 10 + (v & 0x0F).
• Gray code changes only one bit between neighboring values, so a rotary or absolute encoder never reads a wildly wrong value while it is between two positions: gray = n ^ (n >> 1).
• Parity adds one bit so that the number of 1s is even (or odd): it detects any single-bit error.
• Simple checksums (XOR or sum of all bytes, or the two’s complement “LRC” of Modbus ASCII) are cheap but weak: see the comparison with CRCs in Embedded Systems.

n	Binary	Gray code
0	000	000
1	001	001
2	010	011
3	011	010
4	100	110
5	101	111
6	110	101
7	111	100

For the bytes 01 03 00 00 00 02 (a Modbus request) the XOR checksum is 0x00, the sum modulo 256 is 0x06, and the Modbus ASCII LRC (two’s complement of the sum) is 0xFA; the 16-bit CRC used by Modbus RTU (0x0BC4, sent as C4 0B) catches far more errors.

Text: ASCII and UTF-8

ASCII assigns the numbers 0 to 127 to letters, digits, punctuation and control codes: “A” is 0x41, “a” is 0x61 (lower case is upper case + 0x20, one bit), “0” is 0x30 (so a digit character minus 0x30 is its value), newline is 0x0A and carriage return 0x0D. UTF-8 encodes every Unicode character in one to four bytes, keeping ASCII unchanged, and is the default for files, the web and Python 3.

Character	Code point	Bytes in UTF-8	Bytes (hex)
A	U+0041	1	41
é	U+00E9	2	C3 A9
€	U+20AC	3	E2 82 AC
°	U+00B0	2	C2 B0
😀	U+1F600	4	F0 9F 98 80

• Text is bytes plus an encoding. Reading a UTF-8 file as Latin-1 turns “°C” into “°C”. Always say the encoding when you open or decode (open(path, encoding="utf-8"), raw.decode("ascii", errors="replace")).
• The length of a string in bytes is not its length in characters (“°C” is 2 characters and 3 bytes). Fixed-size protocol fields count bytes.
• Line endings: Windows uses CR LF (0x0D 0x0A), Unix LF; many instruments end replies with CR, LF or both, so strip them (strip()).
• Instruments and serial devices are usually plain ASCII; use decode("ascii") and treat anything else as an error.
• Base64 and hex encodings carry binary data through text channels (JSON, e-mail, URLs): hex doubles the size, Base64 increases it by a third.

Reading and writing binary data

A binary file or message is a defined sequence of fields. Read it with the exact layout from the specification: field sizes, signedness, byte order and any padding. Python’s struct module and C’s explicit shifts do this.

struct code	Type	Size
B, b	unsigned / signed byte	1
H, h	unsigned / signed 16-bit	2
I, i	unsigned / signed 32-bit	4
Q, q	unsigned / signed 64-bit	8
f, d	32-bit / 64-bit float	4 / 8
< or > first	little / big-endian, no padding

import struct
HEADER = struct.Struct("<HHIf")                 # 12 bytes, little-endian, no padding
marker, kind, count, scale = HEADER.unpack_from(data, 0)
if marker != 0xCAFE:
    raise ValueError("bad marker: not a frame start")

/* C: do not cast a byte buffer to a struct; read each field explicitly */
uint16_t marker = (uint16_t)(p[0] | (p[1] << 8));
uint32_t count  = (uint32_t)p[4] | ((uint32_t)p[5] << 8) | ((uint32_t)p[6] << 16) | ((uint32_t)p[7] << 24);

• Look at a hex dump when a format is unclear: xxd file.bin | head, hexdump -C, or a hex editor; the first bytes usually give a signature.
• Validate: check lengths against the data you have before you read (a corrupt length field must not make you read past the end), and check a CRC.
• Stream parsers must cope with partial frames: keep a buffer, search for the start marker, and wait for the rest (see Embedded Systems: serial buses).
• NumPy reads arrays of binary values quickly: np.fromfile(path, dtype="<i2") for little-endian 16-bit integers; np.frombuffer for bytes in memory.

Bit-level pitfalls

Pitfall	What happens	Fix
Operator precedence	a & b == c compares first: b == c is evaluated before the AND	Parenthesize: (a & b) == c
Shifting too far	1 << 32 on a 32-bit int is undefined	Use 1u, 1ULL for 64-bit masks, and keep shifts below the width
Signed shifts	Right-shifting a negative value, or left-shifting into the sign bit, is implementation-defined or undefined	Do bit work on unsigned types
~ promotes	~(uint8_t)0x0F becomes a negative int	Cast back: (uint8_t)~x
Bit fields in structs	Layout and order differ between compilers	Use masks and shifts
Wrong byte order	Values look plausible but are scrambled	Check the specification; test with a known value
Octal by accident	A leading zero (010) is octal in C	Write 10, or 0x0A
Reading a register twice	A status read can clear flags on some hardware	Read once into a variable, then test the variable

Related

• Programming Languages: the overview of this guide.
• Numbers and Floating Point: integers, floats and their rounding.
• C and C++ Essentials: types, pointers and structures.
• Bitwise and Register Field Calculator: build masks and decode registers.
• Industrial Communication and Embedded Systems: serial buses: the protocols that carry these bytes.

FAQ

Why is my sensor reading wrong by a large factor?

Most often byte order (high and low bytes swapped), a sign that was not extended, or a field that sits in only some of the bits. Print the raw bytes in hex and compare with the datasheet’s example.

How do I set one bit without changing the others?

OR with a mask to set (reg |= 1 << n), AND with an inverted mask to clear (reg &= ~(1 << n)), XOR to toggle. Never assign the whole register unless you mean to.

What is the difference between & and &&?

& is bitwise AND of the bits of two numbers; && is the logical AND of two truth values and stops early. Mixing them up compiles but gives wrong results.

Why does the same number have different bytes on two computers?

They use different byte orders. Network protocols and file formats define one, and code that reads or writes them must convert explicitly rather than copy memory.

References

• Warren, H. S., Hacker’s Delight, Addison-Wesley (bit tricks and arithmetic).
• IEEE 754-2019; Unicode Standard and RFC 3629 (UTF-8); Python documentation for struct and int.from_bytes.
• Microcontroller reference manuals (register layouts); Modbus Organization, Modbus Application Protocol Specification (register and byte order).

Last reviewed: October 2026. Examples use common formats: always check the specification of your own device.