Information Storage and Retrieval: Structured, Semi-Structured and Unstructured Data

Lesson overview

Explain the main forms of data encountered in storage and retrieval systems and why the choice of structure changes validation, search, analytics and maintenance.

CourseInformation Storage and Retrieval

Topic strandData forms

Assessment styleWorked scenario, applied task and digital exam practice

EvidenceDefinitions, representation, method, result and interpretation

Starter

Write down the user need, the data being stored or searched, and the decision the system has to support. Then predict which representation, index, query method or governance control will matter most in this lesson.

Learning objectives

Distinguish structured data, semi-structured data and unstructured data.
Explain schema-on-write and schema-on-read using realistic retrieval examples.
Choose an appropriate representation for records, documents and media descriptions.

Learning outcomes

Students can distinguish structured data, semi-structured data and unstructured data.
Students can explain schema-on-write and schema-on-read using realistic retrieval examples.
Students can choose an appropriate representation for records, documents and media descriptions.

Key vocabulary, acronyms and terminology

structured data: Data organised in a fixed schema such as tables with typed columns.
semi-structured data: Data with tags, keys or hierarchy but not necessarily fixed relational tables.
unstructured data: Data such as free text, images or audio whose meaning is not arranged as fixed fields.
XML: Extensible Markup Language: a labelled, hierarchical text format often used for semi-structured records.
JSON: JavaScript Object Notation: a lightweight key-value and array format often used for semi-structured web and application data.
HTML: HyperText Markup Language: the markup language used to structure web pages.
OCR: Optical Character Recognition: software extraction of text from scanned images or document pictures.
schema-on-write: The schema is enforced before data is stored.
schema-on-read: Structure is interpreted when the data is queried or analysed.
data model: A chosen way to represent entities, relationships and constraints.

Detailed teaching notes

Core concept

Structured data supports precise validation and querying because the fields and types are known. Semi-structured data, such as XML or JSON, carries its own labels and hierarchy. Unstructured data often needs indexing, metadata or machine analysis before useful retrieval can happen.

Representation choice

A product price table, an XML article record and a scanned poster image all store information, but the retrieval work differs. The table can be filtered by price immediately; the XML can be navigated by tags; the image may need OCR, captions or metadata.

Method and reasoning

Ask what must be validated, what must be searched and what will change over time. Transactional systems usually benefit from schema-on-write. Large mixed collections often use schema-on-read for exploration before a stable model is known.

Risk and limitation

The phrase unstructured can be misleading. Free text and media still have internal structure, but it is not the same as a database schema, so the retrieval system has to extract or attach searchable signals.

Degree-level deep dive

Structure is a retrieval decision

Structured, semi-structured and unstructured data are not value judgements. They describe how much reliable shape the system can depend on when validating, indexing, filtering and analysing the collection.

Schema timing matters

Schema-on-write protects operational integrity before data enters the store. Schema-on-read delays interpretation until analysis, which helps with varied sources but increases the responsibility of the retrieval or analytics layer.

What excellent work shows

A strong answer classifies each data source, identifies what must be validated, and connects the choice to retrieval. For example, orders need fixed fields and constraints, while support transcripts need metadata, text indexing and possibly later feature extraction.

Concrete example to study

One collection, three data forms

Structured: Loan(loan_id, item_id, borrower_id, due_date)
Semi-structured JSON: {"title":"Oral history 12", "speakers":["A. Khan"], "topics":["migration"]}
Unstructured: scanned-poster-1948.jpg

Reasoning

The loan table can enforce types and keys before storage.
The JSON record can vary by interview while still exposing labelled fields for indexing.
The scanned poster needs OCR, image metadata and possibly manual subject tags before it is searchable.

Reveal takeaway

The storage plan should not force one representation onto all data; it should preserve the form that supports validation and retrieval.

Worked example

Scenario

An archive stores book loans, article abstracts and digitised photographs.

Worked solution

Classify book loans as structured because each loan has fixed fields such as borrower, item and due date.
Classify article abstracts as semi-structured if they arrive with tagged title, author, abstract and keyword fields.
Classify photographs as unstructured media for content search, though they may still have structured metadata.
Choose storage: relational tables for loans, XML or JSON records for article descriptions, object storage plus metadata for images.

Reveal model result

The collection needs more than one model. Treating all items as database rows would lose media and document detail; treating everything as unstructured files would make loan integrity and analytics harder.

\text{Define the data, choose the representation, apply the method, interpret the result.}

Applied retrieval task

Create a representation plan for a web publishing archive.

Deliverables

Classify pages, comments, images and access logs by data form.
Decide where schema-on-write is necessary and where schema-on-read is acceptable.
Name one retrieval task supported by each data form.

Success checks

The plan separates data form from storage technology.
The retrieval task is specific enough to test the representation.

Common misconception

A common error is to equate unstructured with useless. Unstructured data can be highly valuable, but it usually needs metadata, extracted features or indexing before it becomes searchable.

Quick checks

1. In this lesson, why does structured data matter?

It replaces the need to understand user tasks or data representation.It is only a presentational label and has no effect on retrieval quality.Data organised in a fixed schema such as tables with typed columns.

2. Which answer best shows degree-level understanding of structured, semi-structured and unstructured data?

Define semi-structured data, apply it to the scenario, then explain the trade-off or risk.Name as many technologies as possible without explaining the scenario.Give a final recommendation without showing how the data or query was represented.

Digital exam practice

Example exam task

A business wants to store customer orders, support-chat transcripts and product images in one searchable platform. In your answer, define the relevant objects or data structures, use course-specific vocabulary, show the method rather than only the result, and finish with a decision about the storage or retrieval system.

Notation and technical toolkit

T(id, title, year)relational table schema Use for structured records with fixed attributes.

<article><title>...</title></article>XML-style hierarchy Use to show semi-structured labelled content.

schema-on-readstructure applied during analysis Use when incoming data is varied or evolving.

What a good answer is expected to show

A strong answer for Structured, Semi-Structured and Unstructured Data the answer should classify each data type, justify where strict structure is needed, and explain why semi-structured or unstructured content still needs metadata for retrieval. It should connect the formal or technical representation to the user's information need instead of listing terms without using them.

How to solve it

Classify each source by its dominant data form.
State which fields need validation before storage.
Explain where schema-on-read is useful for later analytics.
Connect the chosen representation to search and reporting needs.

Model answer

Reveal model answer

Customer orders are structured data because order id, customer id, item id, quantity and date should obey a fixed schema. Support-chat transcripts are mainly unstructured text, although speaker, time and ticket id are structured metadata. Product images are unstructured media, but they need metadata such as product id, alt text and category. I would use schema-on-write for orders to protect integrity, and schema-on-read for exploratory analysis of chats and image annotations.

Practise next

Classify five fields from a streaming-service dataset by data form.
Explain why a JSON log is semi-structured even if different events have different keys.

Self-marking criteria

Defines the scenario objects, data forms or system components before solving.
Uses the lesson vocabulary accurately and in context.
Shows a clear method with enough working for a marker to follow.
Connects the result back to retrieval, storage, analytics or governance.
States a limitation, trade-off or quality risk rather than presenting the answer as absolute.

Extension

Take one real web page and separate its structured metadata, semi-structured HTML and unstructured visible text.

Study route

Save a short worked answer from this lesson using this pattern: define the need or data, choose the representation, show the method, state the result and interpret the implication for the system.

Next lesson: Character Sets, Unicode and Text Representation.