Fundamentals of Computing: Floating Point Representation and Approximation Error

Lesson overview

Explain why real numbers in programs are often approximate.

CourseFundamentals of Computing

Topic strandNumber representation

Assessment styleDigital exam practice

EvidenceWorked answer plus justification

Starter: make the model explicit

Before reading the worked example, write down the objects involved, the claim being made and the notation you expect to use. This prevents the common error of calculating before modelling.

Learning objectives

Describe floating point as sign, scale and significant digits.
Recognise why some decimal fractions cannot be stored exactly.
Choose sensible comparison language for approximate values.

Learning outcomes

By the end of this lesson, you can describe floating point as sign, scale and significant digits.
By the end of this lesson, you can recognise why some decimal fractions cannot be stored exactly.
By the end of this lesson, you can choose sensible comparison language for approximate values.

Key vocabulary

What this lesson is about

Floating point represents real-valued quantities approximately using a sign, a significand and an exponent. It supports very small and very large values, but not every decimal fraction can be stored exactly.

Most real systems use IEEE 754 floating point. That standard gives predictable rules for binary formats, rounding, infinities and NaN values, but it does not make every real number exactly representable.

This lesson is about representation and approximation together: floating point is useful because it scales, but the approximation means equality, rounding and accumulated error must be handled carefully.

Terms, acronyms and named methods

These are the phrases and named techniques used in this lesson. Read this section before the worked example so the notation and examples have a clear meaning.

Floating pointPhrase. A representation for approximate real numbers using a sign, significand and exponent. It is called floating point because the scale changes with the exponent.

SignificandPhrase. The significant digits stored for the number, before scaling by the exponent.

Rounding errorPhrase. The difference between the intended real value and the closest representable stored value.

TolerancePhrase. An acceptable error bound, often written ε, used when comparing approximate values.

IEEE 754Named standard. The common floating-point standard used by most modern hardware and languages. It defines binary formats, rounding behaviour, infinities and NaN values.

Formal notation and definitions

A floating-point value is approximately \pmm \times βᵉ. Rounding error is the difference between the real value and the stored representable value. A finite binary floating-point format has only finitely many significand bits, so many real values are rounded to nearby representable values. Use |x - y| < ε when exact equality is not a safe modelling assumption.

How to read the symbols

$m$ The significand: the stored significant digits.

$\beta$ The base of the floating-point representation, usually 2 in binary hardware.

$e$ The exponent: the scale applied to the significand.

IEEE 754The widely used floating-point standard defining binary formats, rounding behaviour, infinities and NaN values.

$\varepsilon$ A tolerance used when comparing approximate values.

Use the approximation notation to explain why a stored value may be close to, but not exactly equal to, the real number being modelled.

Degree-level reasoning

Representation topics ask you to distinguish mathematical value from machine encoding. The same bit pattern can mean different things under different interpretation rules.

Degree-level answers should mention range, precision, rounding or overflow where those limits affect correctness.

Do not stop at a correct-looking answer. State why the method is valid, whether the result depends on a hidden assumption, and what would count as a counterexample.

Worked formal model

This section shows the model, notation, calculation and interpretation as one worked answer. The notation is part of the reasoning, not decoration.

1. Context and objects

A calculation such as 0.1 + 0.2 can produce a stored value very close to 0.3 without being exactly the decimal number 0.3.

Objects: the real value, the stored representable value, the rounding error, and an acceptable tolerance ε.

Model cue: Use a number line with real values marked continuously and representable floating-point values marked as discrete points. Many real numbers lie between representable values.

2. Mathematical working

\mathrm{stored}(x)=x+\delta\quad\text{where }\delta\text{ is rounding error}

0.1_{10}\text{ has no finite binary expansion, so it is rounded when stored}

\text{IEEE 754 binary formats use finite significand bits, so many real values are rounded}

|x-y|<\varepsilon\quad\text{means x and y are close within tolerance }\varepsilon

Plain text version

A floating-point value is approximately ±m × βᵉ.
Rounding error is the difference between the real value and the stored representable value.
A finite binary floating-point format has only finitely many significand bits, so many real values are rounded to nearby representable values.
Use |x − y| < ε when exact equality is not a safe modelling assumption.

Worked use: In a finite binary floating-point format, the significand has a fixed number of bits. The decimal fraction 0.1 has no finite binary expansion, so IEEE 754 systems round it to a nearby representable value. A program may display a result close to 0.3 after 0.1 + 0.2 because the stored operands and the result were approximations.

3. How to read the working

Read stored(x) = x + δ as a representation statement: the stored value equals the intended real value plus a rounding error.
The symbol δ names the error introduced by rounding to an available floating-point value.
The inequality |x − y| < ε is a comparison rule: it accepts values that are close enough under a stated tolerance.
Interpret ε in context. A graphics coordinate, a bank balance and a scientific simulation may require very different tolerances.

4. Computing meaning and check

The calculation explains why a program may be numerically close but not exact. Correct software should compare approximate values using a tolerance where appropriate.

Now check: Explain why equality tests on floating-point results can be unsafe. Give a condition using an error tolerance ε, describe what ε means, and state when exact integer or decimal arithmetic would be a better model.

Worked example

From scenario to formal reasoning

Scenario: A calculation such as 0.1 + 0.2 can produce a stored value very close to 0.3 without being exactly the decimal number 0.3.

Method: Use the definitions and notation introduced above, then state what the result means in this computing scenario.

Reveal model answer

In a finite binary floating-point format, the significand has a fixed number of bits. The decimal fraction 0.1 has no finite binary expansion, so IEEE 754 systems round it to a nearby representable value. A program may display a result close to 0.3 after 0.1 + 0.2 because the stored operands and the result were approximations.

Worked solution structure

How a strong answer should be written

Define: State the domain and the objects under discussion. For this lesson, begin from A floating-point value is approximately ±m × βᵉ.
Apply: Use the relevant definition from number representation; do not rely on the diagram, wording or intuition alone.
Check: Test a boundary case, counterexample candidate or representation limit.
Conclude: Write one sentence that connects the formal result back to the computing scenario.

Common misconception

A common mistake is calling every unexpected decimal display an arithmetic error. Often the arithmetic is following the representation rules.

Guided practice

Explain why 0.1 has no finite binary expansion and must be rounded in binary floating point.
For a stored value x and expected value y, write a tolerance test using |x - y| < ε.
Choose a sensible ε for a pixel-position comparison and explain why a finance calculation may need a different policy.
Describe how adding many rounded values can accumulate error even when each individual error is small.

Quick checks

1. Why can 0.1 be approximate in binary floating point?

The decimal digit 1 is illegal.Binary cannot store fractions at all.It has no finite binary expansion.

2. What does |x - y| < ε express?

x and y are identical strings.x must be smaller than y.x and y are close within a stated tolerance.

Digital exam practice

Example exam task

Explain why 0.1 + 0.2 may not compare exactly equal to 0.3 in binary floating point. Write a tolerance comparison and justify the tolerance idea for software testing.

Notation toolkit

x+\delta

Means: a stored value may equal the intended value plus a small error

How to use: Use it when explaining rounding error.

|x-y|<\varepsilon

Means: x and y are close within tolerance epsilon

How to use: Use it for safe approximate comparison.

\varepsilon

Means: a chosen tolerance, not a universal constant

How to use: Use it when the acceptable error depends on context.

What a good answer is expected to show

A strong answer for this lesson defines the real value, the stored representable value, the rounding error, and an acceptable tolerance ε, applies the number representation method with visible working, and finishes by interpreting the result in the computing scenario.

How to solve it

Read stored(x) = x + δ as a representation statement: the stored value equals the intended real value plus a rounding error.
The symbol δ names the error introduced by rounding to an available floating-point value.
The inequality |x − y| < ε is a comparison rule: it accepts values that are close enough under a stated tolerance.
Interpret ε in context. A graphics coordinate, a bank balance and a scientific simulation may require very different tolerances.

Model answer

Reveal model answer

\operatorname{stored}(x)=x+\delta\quad\text{where }\delta\text{ is rounding error}

0.1_{10}\notin\{\text{finite binary fractions}\}

|x-0.3|<\varepsilon

Binary floating point stores a rounded representable value using sign, significand and exponent. The decimal 0.1 has no finite binary expansion, so 0.1 and 0.2 are stored approximately and their sum may not be exactly 0.3. A safer comparison is |x - 0.3| < ε, where ε is a context-dependent tolerance.

Practise next

Explain why 0.1 has no finite binary expansion and must be rounded in binary floating point.
For a stored value x and expected value y, write a tolerance test using |x - y| < ε.

Self-marking criteria

Mentions sign, significand and exponent at least once.
Explains that 0.1 has no finite binary expansion.
Uses |x - y| < ε or equivalent.
Explains that ε is context-dependent.
Avoids describing representation error as a processor arithmetic fault.

Extension

Change one assumption in the worked scenario and decide whether the same method still applies. If it does not, name the exact point where the reasoning breaks.

Study route

Save one clean worked answer from this lesson. Include the problem statement, notation, working, final answer and a short note explaining the computing meaning of the result.

Next lesson: Real-Valued Functions and Graph Interpretation.