Statistical Tolerancing and Process Capability

Every process varies, and the variation usually follows a pattern: most parts cluster near the average, with fewer and fewer farther out. If you know the pattern you can ask a more useful question than “is it in tolerance?”: how often will it be out? This page covers the statistics behind that question (from the Machinery’s Handbook chapter on statistical analysis of manufacturing data), the two measures that compare a process to its tolerance (Cp and Cpk), how to set a tolerance from a capability, and how much data you need. The assembly side, using these distributions in a stack-up, is on Tolerance Stack-up Analysis. It is part of the Fits & Tolerances guide.

The normal distribution

The Handbook calls the normal distribution the most widely used statistical distribution for modeling mechanical, physical, electrical and chemical properties that scatter about a well-defined mean value without either positive or negative bias. Its curve is the familiar bell. Two numbers describe it: the mean µ (where it is centered) and the standard deviation σ (how wide it is).

Sample mean: x̄ = (1 ÷ n) Σ x_i
Sample standard deviation: s = √( Σ (x_i − x̄)² ÷ (n − 1) )
Coefficient of variation: V = (s ÷ x̄) × 100%

Within	Share of parts	Outside, per million
± 1σ of the mean	68.26%	317,311
± 2σ	95.44%	45,500
± 2.5758σ	99%	10,001
± 3σ	99.73%	2,700
± 4σ	99.9937%	63.3
± 5σ	99.99994%	0.573
± 6σ	99.9999998%	0.002

The Handbook gives the first three as 68.26% within ±1σ, 95.44% within ±2σ and 99.74% within ±3σ (the last is more precisely 99.73%). A sample’s mean and standard deviation are estimates of the true population values; the more data, the better the estimate (see “How much data you need”). Not all manufacturing data are normal (tool life, for example, is not), and the Handbook lists the other common distributions: log-normal, exponential, binomial, Weibull and Poisson. Check a histogram first.

Cp and Cpk

A capability index compares the spread of a process to the width of its tolerance. USL and LSL are the upper and lower specification limits (the tolerance limits on the drawing).

Cp = (USL − LSL) ÷ 6σ
Cpk = min( (USL − µ) ÷ 3σ , (µ − LSL) ÷ 3σ )

• Cp is what the process could do if it were perfectly centered: the tolerance width divided by the 6σ spread. Cp = 1 means the ±3σ spread just fits the tolerance.
• Cpk is what it is actually doing: the distance from the mean to the nearer limit, in units of 3σ. Cpk equals Cp only when the mean is at the middle of the tolerance, and it falls as the mean moves toward a limit. A negative Cpk means the mean itself is outside the tolerance.
• The gap between Cp and Cpk tells you what to fix: if Cp is good but Cpk is poor, center the process (adjust the setting, the tool offset); if Cp itself is poor, reduce the variation (a better process or machine).
• Cp and Cpk assume a stable (in control) process and a normal distribution. They are meaningless for a drifting process, and misleading for a skewed one.

What a capability means in defects

For a normal process the fraction outside the tolerance follows from the index. This table is for a process centered on the tolerance (Cp = Cpk); the required Cpk values are typical customer requirements.

Cp = Cpk	Share of the tolerance used by the 6σ spread	Tolerance in σ (total width)	Parts out of tolerance (ppm)	Comment
0.67	149%	4.02	44,431	Poor: about 4.4% out
1.00	100%	6	2,700	The ±3σ limits just touch the tolerance
1.33	75%	7.98	66.1	A common minimum for an established process
1.50	67%	9	6.8
1.67	60%	10.02	0.544	A common requirement for critical characteristics
2.00	50%	12	0.002	The “six sigma” capability when centered

A centered Cp of 1.33 means the 6σ spread uses 75% of the tolerance, about 63 parts per million out; 1.67 uses 60% and gives about 0.57 ppm. These match the definitions: 75% and 60%.

When the process is off-center

Off-center, one tail is much larger than the other, and Cpk takes it into account. For a process with Cp = 2.00 (centered, six-sigma capability) whose mean shifts toward a limit, the defects per million are:

Mean shift toward a limit	Cpk	Parts out of tolerance (ppm)
0σ	2.00	0.002
0.5σ	1.83	0.019
1σ	1.67	0.287
1.5σ	1.50	3.4

A 1.5σ shift gives Cpk 1.50 and 3.4 ppm: the “3.4 defects per million” figure in the six-sigma programs, which assumes a long-term 1.5σ drift. That assumption is debated; do not rely on it, and measure your own drift.

Setting a tolerance from capability

A designer can use the capability the process actually has to choose a realistic tolerance, instead of guessing. Rearranging the definition:

Tolerance (total width) = 6 × Cp × σ
Required Cp 1.00: 6σ | 1.33: 8σ | 1.67: 10σ | 2.00: 12σ

So to hold Cp = 1.33 a process with σ = 0.004 mm needs a total tolerance of at least 8 × 0.004 = 0.032 mm (±0.016). If the drawing asks for ±0.01 (a width of 0.02), Cp is 0.02 ÷ (6 × 0.004) = 0.83: about 1.2% of parts would be out even if perfectly centered. Either the tolerance must open up, or the process must improve.

• Use short-term σ (within a lot, one setup) for Cp and Cpk, and long-term σ (across setups, tools, days) for Pp and Ppk. Long-term capability is lower; ask which one a customer’s requirement means.
• Allow for the measurement. The measured scatter includes the gauge’s: σ_observed² = σ_process² + σ_gauge². A poor gauge makes a good process look bad (and hides drift). See the Inspection and Tolerance Stack-up page.
• Put tolerance where it is cheap. A dimension with a capable process can carry a tight tolerance; one that is hard to hold should be given the looser share of an assembly tolerance (see weight factors on the stack-up page).

How much data you need

A mean and standard deviation from a small sample are only estimates. The Handbook gives the absolute error of the sample mean as the standard deviation divided by the square root of the number of points, A_x = s ÷ √n, and the confidence interval for the population mean as x̄ ± t × s ÷ √n, where t comes from Student’s t-distribution (for a 95% two-sided interval, the t_0.025 column, with n − 1 degrees of freedom).

Sample size n	Degrees of freedom	t (95%, two-sided)	Half-width of the 95% interval for the mean
5	4	2.776	1.241 s
10	9	2.262	0.715 s
20	19	2.093	0.468 s
30	29	2.045	0.373 s
50	49	2.010	0.284 s
100	99	1.990	0.199 s

The error in the mean shrinks only with the square root of the sample: four times the data halves the uncertainty. A capability study usually needs at least 30 parts, ideally from several subgroups; Cpk from five parts is mostly noise.

Keeping the process in control

Capability describes a process that is stable. Statistical process control (SPC) is how you check that it stays so: measure small samples at regular intervals, plot their means and ranges on control charts, and look for points beyond the control limits (typically ±3σ of the sample means) or for runs and trends. A process that drifts toward a limit shows up as a falling Cpk long before the first bad part. Control limits describe what the process does; specification limits describe what the customer wants. Never confuse the two.

Worked examples

Example 1: Cp and Cpk from measured data

A 25.000 ± 0.025 mm bore (LSL = 24.975, USL = 25.025) is measured on 20 consecutive parts: 25.006, 25.011, 24.998, 25.004, 25.009, 25.002, 25.013, 24.999, 25.007, 25.005, 25.010, 25.001, 25.008, 25.003, 25.012, 25.000, 25.006, 25.009, 25.004, 25.007.

• Mean x̄ = 25.0057; standard deviation s = 0.0043.
• Cp = 0.050 ÷ (6 × 0.0043) = 1.92.
• Cpu = (25.025 − 25.0057) ÷ (3 × 0.0043) = 1.49; Cpl = (25.0057 − 24.975) ÷ (3 × 0.0043) = 2.36. Cpk = the smaller = 1.49.
• Predicted out of tolerance: about 4.1 ppm, if the process is normal and stays as it is.
• The mean sits 0.0057 above the middle of the tolerance: centering it would lift Cpk to 1.92 (the Cp). With only 20 parts, the 95% interval for the true mean is ± 0.0020 mm.

Example 2: how tight can the tolerance be?

A turning process has a measured σ of 0.004 mm on a diameter. What tolerance does each target Cp support, centered?

Target Cp	Total tolerance = 6 Cp σ	As ± about the middle
1.00	0.024 mm	± 0.012 mm
1.33	0.032 mm	± 0.016 mm
1.67	0.040 mm	± 0.020 mm
2.00	0.048 mm	± 0.024 mm

Example 3: capability of an assembly

Three independent parts each have σ = 0.004 mm. The assembly gap (their sum) has σ = √(3 × 0.004²) = 0.0069 mm, only 1.73 times one part’s, not 3 times. If the gap’s tolerance is ± 0.020 (total 0.040), its capability is Cp = 0.040 ÷ (6 × 0.0069) = 0.96. This is the statistical stack-up of the stack-up page, seen as capability.

Example 4: a drifting process

The same process (σ = 0.006, Cp = 1.39) as its mean drifts upward by tool wear:

Mean drift (mm)	Cpk	ppm out of tolerance
+0.000	1.39	30.9
+0.005	1.11	429
+0.010	0.83	6,210
+0.015	0.56	47,790

A 0.010 mm drift cuts Cpk from 1.39 to about 0.83 and raises the defect rate about two hundredfold, long before a single part is measured out of tolerance. This is why control charts watch the mean.

Troubleshooting

Symptom	Likely cause	Try
Cp good, Cpk poor	The process is off-center	Adjust the setting or offset; check for tool wear or thermal drift
Both low	Too much variation for the tolerance	Improve the process, loosen the tolerance, or add an operation
Cpk changes a lot between studies	Too few parts, or unstable process, or a different σ (short vs long term)	Use at least 30 parts; check the control chart; state short- or long-term
Predicted ppm is zero but defects occur	Data are not normal (skewed, bimodal, or truncated by sorting)	Plot a histogram; use a non-normal method or measure the tails
Process looks worse than it is	Gauge variation is a large share of the tolerance	Do a gauge study; improve the measurement system

Related

• Fits & Tolerances: the overview and quick reference.
• Tolerance Stack-up Analysis: using distributions in an assembly.
• Tolerances by Process and Surface Finish: typical process capability.
• Process Capability Calculator: Cp, Cpk and defect rates from data or from σ.

FAQ

What is a good Cpk?

A common minimum for an established process is 1.33, and 1.67 for critical characteristics, but the requirement belongs to the customer or the quality system. For a new process, many require a higher value (such as 1.67) until the process is proven.

What is the difference between Cp and Pp?

Cp and Cpk use the short-term (within-subgroup) standard deviation, a measure of what the process can do when running well. Pp and Ppk use the overall (long-term) standard deviation, which includes drift between setups and shifts. Pp is always no better than Cp.

Can I compute Cpk from a handful of parts?

You can, but the result will be uncertain. With n = 5 the 95% interval on the mean is more than 1.2 standard deviations wide each way. Use 30 or more parts, and report the number you used.

Why is σ for an RSS stack-up taken as range ÷ 6?

Because it assumes a Cp of 1.0: the tolerance width equals 6σ. A process with Cp = 1.33 has σ = range ÷ 8, so using ÷ 6 is conservative for a capable process.

Does a tight tolerance always cost more?

Usually, because a tighter tolerance needs a more accurate process or more inspection. But the cost depends on the process: a tolerance within what a process holds easily costs nothing extra. See the process page.

What if my data are not normal?

Plot them. Skewed or bounded data (flatness, runout and positions, which cannot be negative) are not normal. Use the methods for non-normal data, or Monte Carlo with the real distribution.

References

• Machinery’s Handbook, 31st edition, Industrial Press: “Statistical Analysis of Manufacturing Data” (statistics theory, probability, normal distribution analysis, confidence intervals, Student’s t) and “Tolerance Analysis and Assignment”.
• AIAG, Statistical Process Control (SPC) Reference Manual (capability and performance indices, control charts).
• ISO 22514-2, Statistical methods in process management: capability and performance.

Last reviewed: October 2026. Defect rates assume a normal distribution; check the shape of your own data.