Information Storage and Retrieval: Multimedia Representation: Images, Audio and Video

Lesson overview

Explain how image, audio and video content is represented for storage and retrieval, including sampling, compression, codecs, bit rate and metadata extraction.

CourseInformation Storage and Retrieval

Topic strandData representation

Assessment styleWorked scenario, applied task and digital exam practice

EvidenceDefinitions, representation, method, result and interpretation

Starter

Write down the user need, the data being stored or searched, and the decision the system has to support. Then predict which representation, index, query method or governance control will matter most in this lesson.

Learning objectives

Describe raster image representation using pixels, colour depth and compression.
Explain audio sampling rate, bit depth and compression trade-offs.
Explain why video storage depends on frames, codecs, bit rate and metadata.

Learning outcomes

Students can describe raster image representation using pixels, colour depth and compression.
Students can explain audio sampling rate, bit depth and compression trade-offs.
Students can explain why video storage depends on frames, codecs, bit rate and metadata.

Key vocabulary, acronyms and terminology

raster image: An image represented as a grid of pixels.
sampling rate: The number of audio samples captured per second.
codec: An encoder-decoder method for compressing and decompressing media.
bit rate: The number of bits used per second of media.
lossy compression: Compression that removes information to reduce size.
OCR: Optical Character Recognition: extraction of text from images or scanned documents so it can be indexed.
thumbnail: A smaller representation used for browsing and preview.

Detailed teaching notes

Core concept

Multimedia retrieval begins with representation. An image, audio file or video is not directly searchable like a table; the system needs technical metadata, descriptive metadata and sometimes extracted features such as speech transcripts or visual labels.

Representation choice

Images are often represented as pixels plus metadata, audio as sampled waveforms or compressed streams, and video as frame sequences compressed by a codec. Storage decisions affect retrieval speed, preview quality and network delivery.

Method and reasoning

For each media type, separate the original asset, derivative files and searchable descriptions. A video archive might store a master file, streaming transcodes, thumbnails, transcript text and rights metadata.

Risk and limitation

Compression choices can destroy information needed later. A low bit rate may be fine for casual viewing but unsuitable for forensic analysis, accessibility captions or high-quality preservation.

Degree-level deep dive

Media needs derived evidence

Images, audio and video are stored as technical media objects, but retrieval usually depends on derived representations: thumbnails, transcripts, captions, tags, timestamps, technical metadata and rights metadata.

Compression has consequences

Codec, resolution, sampling rate and bit rate choices affect later use. A compressed streaming copy may be suitable for playback but not for preservation, accessibility work, forensic inspection or high-quality re-use.

What excellent work shows

A strong answer separates the master asset, derivative files and searchable descriptions. It explains which metadata is indexed, which file is displayed, and which representation supports accessibility, browsing and evidence-based retrieval.

Concrete example to study

Lecture video asset package

master.mp4: high-quality preservation copy
stream-720p.mp4: compressed delivery copy
thumbnail-001.jpg: browsing preview
transcript.vtt: searchable spoken text
metadata.json: module, lecturer, date, rights, captions_status

Reasoning

The master file is kept for preservation, not necessarily streamed to every student.
The transcript becomes searchable text, while thumbnails support browsing.
Rights and captions metadata control access, reuse and accessibility.

Reveal takeaway

A multimedia archive is a bundle of representations; the video file alone is not enough for good retrieval.

Worked example

Scenario

A lecture capture system must store searchable video recordings.

Worked solution

Store the video using an appropriate codec and bit rate for streaming.
Generate thumbnails and preview clips for browsing.
Create speech-to-text transcripts so spoken content becomes searchable text.
Attach metadata such as module, date, speaker, rights and captions status.

Reveal model result

The retrieval system should search transcripts and metadata, not only filenames. The media file remains the evidence, but searchable text and previews make it discoverable.

\text{Define the data, choose the representation, apply the method, interpret the result.}

Applied retrieval task

Plan storage for a small multimedia teaching archive.

Deliverables

Choose representations for images, audio and video.
List derivative files needed for browsing or accessibility.
Identify which metadata should be indexed.

Success checks

The plan explains the retrieval role of each derivative.
Accessibility is included, not treated as optional decoration.

Common misconception

A filename is not enough metadata for multimedia retrieval. Searchable access usually depends on transcripts, tags, timestamps, thumbnails and rights information.

Quick checks

1. In this lesson, why does raster image matter?

It is only a presentational label and has no effect on retrieval quality.It replaces the need to understand user tasks or data representation.An image represented as a grid of pixels.

2. Which answer best shows degree-level understanding of multimedia representation: images, audio and video?

Give a final recommendation without showing how the data or query was represented.Define sampling rate, apply it to the scenario, then explain the trade-off or risk.Name as many technologies as possible without explaining the scenario.

Digital exam practice

Example exam task

A university wants students to search inside recorded lectures and browse visual summaries before opening a full video. In your answer, define the relevant objects or data structures, use course-specific vocabulary, show the method rather than only the result, and finish with a decision about the storage or retrieval system.

Notation and technical toolkit

size = width x height x depthrough raster storage estimate Use to reason about image storage before compression.

samples/secaudio sampling rate Use to compare audio fidelity and file size.

bit ratebits per second of encoded media Use to estimate streaming and storage requirements.

What a good answer is expected to show

A strong answer for Multimedia Representation: Images, Audio and Video the answer should cover codec and bit rate decisions, transcript-based retrieval, thumbnails for browsing and metadata for filtering. It should connect the formal or technical representation to the user's information need instead of listing terms without using them.

How to solve it

Separate storage of the video asset from retrieval signals.
Identify technical choices such as codec, resolution and bit rate.
Add derived text and thumbnails for search and browsing.
State metadata needed for filtering, rights and accessibility.

Model answer

Reveal model answer

I would store a high-quality master recording and create streaming versions with suitable codec, resolution and bit rate. For retrieval, the system should index lecture title, module, speaker, date and speech-to-text transcript. Thumbnails or scene previews support browsing before playback, while captions improve accessibility and search. The answer should not claim that video is searchable merely because it is stored; searchable representations must be extracted or attached.

Practise next

Estimate the uncompressed size of a tiny raster image before explaining why compression is used.
List metadata fields needed to retrieve podcast episodes by speaker and topic.

Self-marking criteria

Defines the scenario objects, data forms or system components before solving.
Uses the lesson vocabulary accurately and in context.
Shows a clear method with enough working for a marker to follow.
Connects the result back to retrieval, storage, analytics or governance.
States a limitation, trade-off or quality risk rather than presenting the answer as absolute.

Extension

Compare lossy and lossless compression for a museum image archive and a streaming entertainment service.

Study route

Save a short worked answer from this lesson using this pattern: define the need or data, choose the representation, show the method, state the result and interpret the implication for the system.

Next lesson: Metadata and Descriptive Records.