Machinist Decimal Reference

Everything a machinist needs in decimal-inch form: thou and tenths definitions, surface finish units, standard tolerance bands, and the conversion between fractional shop talk and decimal print talk.

Decimal inches are the working language of a machine shop because they match the readout on digital calipers, micrometers, and CNC control panels. This page covers the vocabulary and the arithmetic; for a single conversion, use the inch to decimal calculator or the decimal to fraction calculator. For more on precision and how measuring tools display decimal inches, see the decimal inch measurement guide, or the full inch conversion table for every fraction at once.

• Thou / mil: 0.001 inch (one thousandth), equal to 0.0254 mm. The standard unit for everyday shop tolerances.
• Tenth: 0.0001 inch (one ten-thousandth), equal to 0.00254 mm. This is the precision of a good tenths indicator and the level toolmakers and grinders work to.
• Microinch (μin): 0.000001 inch, equal to 0.0000254 mm. Used almost exclusively for surface finish (Ra) callouts, not for size tolerances.
• Decimal foot: 1 inch equals 1/12 foot, or 0.0833 decimal feet. Useful when a drawing mixes feet and inch dimensions.

• Drilled hole: ±0.005" (±0.13 mm), the loosest common machining tolerance, typical of a twist drill without reaming.
• Reamed hole: ±0.001" (±0.025 mm), used when a hole needs to hold a dowel pin or bushing precisely.
• Milled feature: ±0.002" (±0.05 mm), a realistic general-purpose tolerance for a three-axis mill on rigid stock.
• Ground surface: ±0.0005" (±0.013 mm), achievable on a surface or cylindrical grinder in a stable shop.
• Lapped / honed: ±0.0001" (±0.0025 mm), reserved for gage blocks, bearing bores, and other precision fits.

Engineering prints dimension in decimal inches. A callout like "0.250 ±0.005" means the feature must measure between 0.245 and 0.255 inches, and any reading outside that band is a reject. Fractional callouts, like 1/4", imply a looser, unstated tolerance, usually around ±1/64" (0.015625") unless the title block specifies a general tolerance table.

A shop drawing calls for a slot at 3/8" ± 1/64". Convert both numbers to decimal first: 3/8" = 0.375" and 1/64" = 0.015625". Add and subtract to get the acceptable range: 0.375" − 0.015625" = 0.359375" on the low side, and 0.375" + 0.015625" = 0.390625" on the high side.

Any caliper reading between 0.359375" and 0.390625" is a passing part. In metric terms that band runs from about 9.13 mm to 9.92 mm (each limit multiplied by 25.4). Working the conversion once, before cutting, avoids doing fraction math at the machine every time a part is checked.

Surface finish (roughness) is a separate specification from size and is almost always given in microinches Ra in the United States, or micrometers Ra on metric prints. As-milled or as-turned surfaces typically fall around 63 to 125 μin Ra, a fine-finish end mill pass can get down to 32 μin Ra, and grinding routinely reaches 8 to 16 μin Ra. Lapping, honing, and polishing push below 4 μin Ra for optical or hydraulic-seal surfaces.

To convert microinches to millimeters, multiply by 0.0000254 (since 1 inch = 25.4 mm and a microinch is one-millionth of an inch). A 32 μin Ra surface is therefore about 0.0008128 mm Ra, a number small enough that shops almost never convert it by hand and instead compare directly against a surface comparator or profilometer trace calibrated in microinches.