CloudAcademia
EN

Cloud Computing · Module 08 · Cloud Storage: Block, File and Object

Cloud Storage: Block, File and Object

Lesson50 minBeginnerMarkdown

Showing raw Markdown: the renderer could not load.

## Why this matters
A small team launches a photo-sharing app. They keep the photos on the disk of the virtual machine that runs the app, because that is the easiest thing to do. Three months later the app needs a second server, and the second server can't see the first one's photos. Then the disk fills up. Then a developer runs a cleanup script against the wrong folder, and there is no copy to restore from.

Each problem has a different storage answer. Photos belong in **object storage**, which any number of servers can reach over the network and which grows without you planning for it. The database needs a fast **block volume**. The shared configuration folder fits a **file share**. And none of it is safe until you have versioning, backups and encryption in place. Cloud providers sell all of these under different brand names. This lesson gives you the vendor-neutral model behind them, so you can pick the right one and avoid the classic mistakes: confusing durability with backup, and leaving a bucket public.

## Learning objectives
- Compare block, file and object storage in access method, performance, sharing and typical use cases.
- Explain durability versus availability, how replication and erasure coding protect data, and what storage classes trade off.
- Use object storage through the S3 API: create buckets, upload and list objects, enable versioning and add lifecycle rules.
- Share an object safely with a pre-signed URL and explain why public buckets are risky.
- Explain snapshots, backups and encryption at rest and in transit, and who manages the keys.

## Key concepts

### Three kinds of storage: block, file and object
Think of three ways to keep paper documents. **Block storage** is an empty filing cabinet: you decide how to label and arrange everything. **File storage** is a shared office cabinet with labelled drawers that everyone on the team can open. **Object storage** is a warehouse with a clerk: you hand over a box, get a ticket number, and later present the ticket to get the box back.

**Block storage** exposes a **volume**: a fixed-size array of numbered blocks that a server sees as a raw disk, such as `/dev/vdb` on Linux. The operating system puts a **filesystem** (ext4, XFS, NTFS) on it, or a database uses it directly. You attach a volume to one instance at a time (some providers offer a multi-attach mode, but then you need a cluster-aware filesystem, or you corrupt the data). Block storage is judged on three numbers:

- **IOPS** (input/output operations per second): how many reads and writes per second.
- **Throughput**: how many bytes per second. It equals IOPS multiplied by the size of each operation. For example, 3,000 IOPS of 16 KiB each is 48,000 KiB/s, about 47 MiB/s.
- **Latency**: how long one operation takes, usually under a few milliseconds.

**File storage** is a shared filesystem served over the network. Many servers mount the same share at the same time using the **Network File System (NFS)**, common on Linux (NFSv4.1 is defined in RFC 8881), or the **Server Message Block (SMB)** protocol, common on Windows. The service runs the filesystem, so you get directories, permissions, file locking and **POSIX semantics** (the Portable Operating System Interface rules for files: you can append to a file, rename it atomically, change one byte in the middle, and lock a region). That makes file storage the easy choice for software written for ordinary folders: home directories, content-management systems and shared build artefacts.

**Object storage** stores **objects** (data plus metadata) in **buckets**. You never mount it: applications talk to it through an application programming interface (API) over the Hypertext Transfer Protocol (HTTP), using requests such as `PUT`, `GET`, `LIST` and `DELETE`. There are no directories to maintain and no real limit on the number of objects, which is why it scales out across thousands of machines and costs less per gigabyte than the other two. The price is the access model: an object is written and read as a whole. You can't change one byte in place or rename an object; you upload a new copy.

<figure class="diagram" style="--dg-w: 1100px"><svg xmlns="http://www.w3.org/2000/svg" class="dg" viewBox="0 0 1100 420" width="1100" height="420" role="img" aria-labelledby="storage-access-paths-title storage-access-paths-desc" font-family="Inter, system-ui, -apple-system, Segoe UI, Roboto, sans-serif" xml:lang="en">
<title id="storage-access-paths-title">Block, file and object storage: how each is accessed</title>
<desc id="storage-access-paths-desc">Three columns. Block storage: one instance mounts a volume as a block device, for example /dev/vdb on /data, and talks to it in blocks at an offset; the volume holds raw blocks, for example 100 GiB, and the instance creates the filesystem. Chips: IOPS, latency, databases. File storage: many instances mount the same share, for example //files/share, over NFS or SMB using files and paths; the file share holds directories and permissions and the service runs the filesystem. Chips: shared, POSIX, home folders. Object storage: any application with credentials, anywhere on the network, sends HTTPS requests PUT, GET and LIST to a bucket that maps a key to an object plus metadata in a flat namespace with no filesystem. Chips: scales out, cheap, backups and media.</desc>
<rect class="dg-bg" x="0.5" y="0.5" width="1099" height="419" rx="16" fill="#ffffff" stroke="#e3e3dd"/>
<g class="dg-l1"><rect class="dg-tint" x="20" y="20" width="340" height="380" rx="16" fill="#f0f6f9" stroke="#9ac3d9" stroke-width="1.2" stroke-dasharray="6 5"/>
<g class="dg-icon" transform="translate(36 33) scale(0.7500)" fill="none" stroke="#0369a1" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><ellipse cx="12" cy="5" rx="9" ry="3"/><path d="M3 5V19A9 3 0 0 0 21 19V5"/><path d="M3 12A9 3 0 0 0 21 12"/></g>
<text x="62" y="47" font-size="13" class="dg-ink" fill="#0369a1" font-weight="700">Block storage</text></g>
<rect class="dg-card" x="40" y="64" width="300" height="96" rx="12" fill="#ffffff" stroke="#cfcfc8"/>
<g class="dg-icon-neutral" transform="translate(56 80) scale(1.2500)" fill="none" stroke="#17171c" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><rect width="20" height="8" x="2" y="2" rx="2"/><rect width="20" height="8" x="2" y="14" rx="2"/><path d="M6 6h.01"/><path d="M6 18h.01"/></g>
<text x="98" y="101" font-size="15" class="dg-text" fill="#17171c" font-weight="700">Instance</text>
<text x="58" y="132" font-size="12.5" class="dg-text" fill="#17171c" font-weight="500" font-family="JetBrains Mono, ui-monospace, Menlo, Consolas, monospace">mounts /dev/vdb on /data</text>
<text x="58" y="152" font-size="13" class="dg-muted" fill="#5c5c68" font-weight="500">one instance at a time</text>
<rect class="dg-card" x="40" y="248" width="300" height="96" rx="12" fill="#ffffff" stroke="#cfcfc8"/>
<g class="dg-l1"><g class="dg-icon" transform="translate(56 264) scale(1.2500)" fill="none" stroke="#0369a1" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><ellipse cx="12" cy="5" rx="9" ry="3"/><path d="M3 5V19A9 3 0 0 0 21 19V5"/><path d="M3 12A9 3 0 0 0 21 12"/></g></g>
<g class="dg-l1"><text x="98" y="285" font-size="15" class="dg-ink" fill="#0369a1" font-weight="700">Volume</text></g>
<text x="58" y="316" font-size="13" class="dg-muted" fill="#5c5c68" font-weight="500">raw blocks, e.g. 100 GiB</text>
<text x="58" y="336" font-size="13" class="dg-muted" fill="#5c5c68" font-weight="500">you create the filesystem</text>
<g class="dg-l1"><path class="dg-stroke" d="M190 162V240" stroke="#0369a1" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-l1"><path class="dg-stroke" d="M184 235l6 8 6-8" stroke="#0369a1" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-l1"><rect class="dg-bg-plain" x="81.18" y="190" width="217.64" height="24" rx="12" fill="#ffffff"/><text x="190" y="206.5" font-size="11.5" class="dg-ink" fill="#0369a1" font-weight="600" text-anchor="middle">blocks: read/write at an offset</text></g>
<g class="dg-l1"><rect class="dg-tint" x="40" y="356" width="53" height="26" rx="13" fill="#e6f0f6" stroke="#8ebcd5" stroke-width="1.2"/><text x="66.5" y="373.5" font-size="12" class="dg-ink" fill="#0369a1" font-weight="700" text-anchor="middle">IOPS</text></g>
<g class="dg-l1"><rect class="dg-tint" x="99" y="356" width="74" height="26" rx="13" fill="#e6f0f6" stroke="#8ebcd5" stroke-width="1.2"/><text x="136" y="373.5" font-size="12" class="dg-ink" fill="#0369a1" font-weight="700" text-anchor="middle">latency</text></g>
<g class="dg-l1"><rect class="dg-tint" x="179" y="356" width="89" height="26" rx="13" fill="#e6f0f6" stroke="#8ebcd5" stroke-width="1.2"/><text x="223.5" y="373.5" font-size="12" class="dg-ink" fill="#0369a1" font-weight="700" text-anchor="middle">databases</text></g>
<g class="dg-l2"><rect class="dg-tint" x="380" y="20" width="340" height="380" rx="16" fill="#f1f7f3" stroke="#a1ccb1" stroke-width="1.2" stroke-dasharray="6 5"/>
<g class="dg-icon" transform="translate(396 33) scale(0.7500)" fill="none" stroke="#15803d" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M15 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V7Z"/><path d="M14 2v4a2 2 0 0 0 2 2h4"/><path d="M10 9H8"/><path d="M16 13H8"/><path d="M16 17H8"/></g>
<text x="422" y="47" font-size="13" class="dg-ink" fill="#15803d" font-weight="700">File storage</text></g>
<rect class="dg-card" x="400" y="64" width="300" height="96" rx="12" fill="#ffffff" stroke="#cfcfc8"/>
<g class="dg-icon-neutral" transform="translate(416 80) scale(1.2500)" fill="none" stroke="#17171c" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><rect width="10" height="14" x="3" y="8" rx="2"/><path d="M5 4a2 2 0 0 1 2-2h12a2 2 0 0 1 2 2v12a2 2 0 0 1-2 2h-2.4"/><path d="M8 18h.01"/></g>
<text x="458" y="101" font-size="15" class="dg-text" fill="#17171c" font-weight="700">Instances</text>
<text x="418" y="132" font-size="12.5" class="dg-text" fill="#17171c" font-weight="500" font-family="JetBrains Mono, ui-monospace, Menlo, Consolas, monospace">mount //files/share</text>
<text x="418" y="152" font-size="13" class="dg-muted" fill="#5c5c68" font-weight="500">many at once</text>
<rect class="dg-card" x="400" y="248" width="300" height="96" rx="12" fill="#ffffff" stroke="#cfcfc8"/>
<g class="dg-l2"><g class="dg-icon" transform="translate(416 264) scale(1.2500)" fill="none" stroke="#15803d" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><path d="M15 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V7Z"/><path d="M14 2v4a2 2 0 0 0 2 2h4"/><path d="M10 9H8"/><path d="M16 13H8"/><path d="M16 17H8"/></g></g>
<g class="dg-l2"><text x="458" y="285" font-size="15" class="dg-ink" fill="#15803d" font-weight="700">File share</text></g>
<text x="418" y="316" font-size="13" class="dg-muted" fill="#5c5c68" font-weight="500">directories and permissions</text>
<text x="418" y="336" font-size="13" class="dg-muted" fill="#5c5c68" font-weight="500">the service runs the filesystem</text>
<g class="dg-l2"><path class="dg-stroke" d="M550 162V240" stroke="#15803d" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-l2"><path class="dg-stroke" d="M544 235l6 8 6-8" stroke="#15803d" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-l2"><rect class="dg-bg-plain" x="457.28" y="190" width="185.44" height="24" rx="12" fill="#ffffff"/><text x="550" y="206.5" font-size="11.5" class="dg-ink" fill="#15803d" font-weight="600" text-anchor="middle">NFS · SMB: files and paths</text></g>
<g class="dg-l2"><rect class="dg-tint" x="400" y="356" width="67" height="26" rx="13" fill="#e8f2ec" stroke="#96c6a8" stroke-width="1.2"/><text x="433.5" y="373.5" font-size="12" class="dg-ink" fill="#15803d" font-weight="700" text-anchor="middle">shared</text></g>
<g class="dg-l2"><rect class="dg-tint" x="473" y="356" width="60" height="26" rx="13" fill="#e8f2ec" stroke="#96c6a8" stroke-width="1.2"/><text x="503" y="373.5" font-size="12" class="dg-ink" fill="#15803d" font-weight="700" text-anchor="middle">POSIX</text></g>
<g class="dg-l2"><rect class="dg-tint" x="539" y="356" width="110" height="26" rx="13" fill="#e8f2ec" stroke="#96c6a8" stroke-width="1.2"/><text x="594" y="373.5" font-size="12" class="dg-ink" fill="#15803d" font-weight="700" text-anchor="middle">home folders</text></g>
<g class="dg-l4"><rect class="dg-tint" x="740" y="20" width="340" height="380" rx="16" fill="#fbf4f0" stroke="#e7b39e" stroke-width="1.2" stroke-dasharray="6 5"/>
<g class="dg-icon" transform="translate(756 33) scale(0.7500)" fill="none" stroke="#c2410c" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M17.5 19H9a7 7 0 1 1 6.71-9h1.79a4.5 4.5 0 1 1 0 9Z"/></g>
<text x="782" y="47" font-size="13" class="dg-ink" fill="#c2410c" font-weight="700">Object storage</text></g>
<rect class="dg-card" x="760" y="64" width="300" height="96" rx="12" fill="#ffffff" stroke="#cfcfc8"/>
<g class="dg-icon-neutral" transform="translate(776 80) scale(1.2500)" fill="none" stroke="#17171c" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><path d="M20 16V7a2 2 0 0 0-2-2H6a2 2 0 0 0-2 2v9m16 0H4m16 0 1.28 2.55a1 1 0 0 1-.9 1.45H3.62a1 1 0 0 1-.9-1.45L4 16"/></g>
<text x="818" y="101" font-size="15" class="dg-text" fill="#17171c" font-weight="700">Applications</text>
<text x="778" y="132" font-size="12.5" class="dg-text" fill="#17171c" font-weight="500" font-family="JetBrains Mono, ui-monospace, Menlo, Consolas, monospace">any client with credentials</text>
<text x="778" y="152" font-size="13" class="dg-muted" fill="#5c5c68" font-weight="500">anywhere on the network</text>
<rect class="dg-card" x="760" y="248" width="300" height="96" rx="12" fill="#ffffff" stroke="#cfcfc8"/>
<g class="dg-l4"><g class="dg-icon" transform="translate(776 264) scale(1.2500)" fill="none" stroke="#c2410c" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><path d="M21 8a2 2 0 0 0-1-1.73l-7-4a2 2 0 0 0-2 0l-7 4A2 2 0 0 0 3 8v8a2 2 0 0 0 1 1.73l7 4a2 2 0 0 0 2 0l7-4A2 2 0 0 0 21 16Z"/><path d="m3.3 7 8.7 5 8.7-5"/><path d="M12 22V12"/></g></g>
<g class="dg-l4"><text x="818" y="285" font-size="15" class="dg-ink" fill="#c2410c" font-weight="700">Bucket</text></g>
<text x="778" y="316" font-size="13" class="dg-muted" fill="#5c5c68" font-weight="500">key → object + metadata</text>
<text x="778" y="336" font-size="13" class="dg-muted" fill="#5c5c68" font-weight="500">flat namespace, no filesystem</text>
<g class="dg-l4"><path class="dg-stroke" d="M910 162V240" stroke="#c2410c" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-l4"><path class="dg-stroke" d="M904 235l6 8 6-8" stroke="#c2410c" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-l4"><rect class="dg-bg-plain" x="826.94" y="190" width="166.12" height="24" rx="12" fill="#ffffff"/><text x="910" y="206.5" font-size="11.5" class="dg-ink" fill="#c2410c" font-weight="600" text-anchor="middle">HTTPS: PUT · GET · LIST</text></g>
<g class="dg-l4"><rect class="dg-tint" x="760" y="356" width="96" height="26" rx="13" fill="#f9ece7" stroke="#e4aa92" stroke-width="1.2"/><text x="808" y="373.5" font-size="12" class="dg-ink" fill="#c2410c" font-weight="700" text-anchor="middle">scales out</text></g>
<g class="dg-l4"><rect class="dg-tint" x="862" y="356" width="60" height="26" rx="13" fill="#f9ece7" stroke="#e4aa92" stroke-width="1.2"/><text x="892" y="373.5" font-size="12" class="dg-ink" fill="#c2410c" font-weight="700" text-anchor="middle">cheap</text></g>
<g class="dg-l4"><rect class="dg-tint" x="928" y="356" width="125" height="26" rx="13" fill="#f9ece7" stroke="#e4aa92" stroke-width="1.2"/><text x="990.5" y="373.5" font-size="12" class="dg-ink" fill="#c2410c" font-weight="700" text-anchor="middle">backups, media</text></g>
</svg></figure>

*Figure 1. The three storage types differ in who runs the filesystem and in how a client reaches the data: blocks, files and paths, or HTTP requests for whole objects.*

| | Block | File | Object |
|---|---|---|---|
| Access method | Raw blocks through the OS (`/dev/vdb`) | NFS or SMB mount | HTTP API (S3-style) |
| Unit of data | Block (for example 4 KiB) | File in a directory | Whole object, up to a provider limit |
| Sharing | One instance at a time | Many instances | Any client with credentials |
| Update in place | Yes | Yes | No: replace the whole object |
| Strengths | Lowest latency, highest IOPS | Familiar folders, POSIX | Scale, durability, low cost per GB |
| Typical use | Databases, boot disks | Shared folders, legacy applications | Photos, backups, logs, static websites, data lakes |
| Cloud examples | Amazon EBS, Azure Managed Disks, Google Persistent Disk | Amazon EFS, Azure Files, Google Filestore | Amazon S3, Azure Blob Storage, Google Cloud Storage |

> [!CHECK]
> A database server needs the lowest possible latency and does many small random writes. Which storage type fits best?
>
> - [ ] Object storage, because it scales without limit
> - [ ] File storage, because several servers can mount it at once
> - [x] A block volume attached to the database instance
> - [ ] Any of them: the type only changes the price
>
> Block volumes give direct, low-latency access at an offset and are sized for IOPS. Object storage replaces whole objects over HTTP, and a network file share adds protocol overhead and sharing features a single database doesn't need.

### Object storage: buckets, keys and the S3 API
A **bucket** is a named container for objects. An object has a **key** (its name), the data (a sequence of bytes) and **metadata**: system fields such as size, content type and last-modified time, plus custom key-value pairs you add, such as `project=apollo`. Bucket names are unique within a provider (or a region, depending on the provider), because they usually appear in a URL.

The namespace is **flat**: a bucket holds keys, not folders. A key such as `reports/2026/q3.txt` is one string that happens to contain slashes. The part up to a slash is called a **prefix**, and listing with a delimiter (`/`) lets tools and consoles *show* folders. But there is no `reports/` directory object to create, rename or fill: if you delete the last object under a prefix, the "folder" disappears. Renaming a "folder" means copying and deleting every object under the prefix.

Applications reach objects with plain HTTP. A request to upload and one to download look like this (the host name is an example):

```text
PUT /reports/2026/q3.txt HTTP/1.1
Host: lab-docs.objects.example.com
Content-Type: text/plain
x-amz-meta-project: apollo
Authorization: <signature computed from your secret key>

Quarterly report v1
```

```text
GET /reports/2026/q3.txt HTTP/1.1
Host: lab-docs.objects.example.com
Authorization: <signature computed from your secret key>
```

Most providers accept the bucket either in the host name (`lab-docs.objects.example.com`, "virtual-hosted style") or as the first path segment (`objects.example.com/lab-docs/…`, "path style"). The `Authorization` header carries a signature made with your access key, so the service knows who is asking and nobody else can reuse the request. Large objects are uploaded in parts (a **multipart upload**), which can be retried independently.

The **S3 API** is the HTTP interface of Amazon Simple Storage Service (S3). It has become the de facto standard for object storage: many providers (for example Backblaze B2, Cloudflare R2, Wasabi, and Google Cloud Storage through its interoperability mode) and open-source servers (Ceph RADOS Gateway, SeaweedFS, Garage) implement it, and so do emulators for testing, such as Moto, which you use in the lab. The same SDKs and command-line tools work against all of them; you change only the endpoint URL and the keys. Each implementation covers a different subset of features, so test the ones you depend on.

> [!NOTE]
> "S3-compatible" is a claim, not a certification. Core calls (create bucket, put, get, list, delete) work almost everywhere. Versioning, lifecycle rules, bucket policies, encryption options and event notifications differ between implementations. Check the provider's compatibility page before you depend on a feature.

Many modern object stores, including Amazon S3 since December 2020, give **strong read-after-write consistency**: once a `PUT` succeeds, the next `GET` or `LIST` returns the new data. Older systems and some compatible servers were only eventually consistent, so check this too.

> [!CHECK]
> A bucket contains the keys `logs/2026/01.txt` and `logs/2026/02.txt`. You delete both objects. What happens to the `logs/` folder?
>
> - [ ] It stays as an empty folder, and you must delete it separately
> - [ ] The delete fails, because a folder isn't empty
> - [x] Nothing remains: `logs/` was only a shared prefix in the key names, not an object
> - [ ] It moves to a trash folder inside the bucket
>
> Object storage has a flat namespace. Slashes are ordinary characters in the key, and consoles only draw folders by grouping keys on a common prefix.

### Durability, availability and how data is protected
Two words that sound alike promise different things.

- **Durability** is the probability that data is not lost or corrupted over time.
- **Availability** is the fraction of time the service answers requests.

Providers often state durability as "eleven nines": 99.999999999%. This is usually an annual figure per object. The chance of losing any one object in a year is about 1 in 100 billion, so if you stored 10 million objects you would expect to lose one roughly every 10,000 years. Availability is far lower and is a different promise: 99.9% allows about 8.8 hours of downtime a year, 99.99% about 53 minutes.

What eleven nines **does not** mean:

- It doesn't mean the data is reachable right now. An outage makes data unavailable but not lost.
- It doesn't protect you from yourself: if you or an attacker delete an object, overwrite it or delete the bucket, the service durably keeps the deletion.
- It is a design target, backed by a statistical model of disk failures, not a contractual guarantee. Service-level agreements (SLAs) usually offer service credits for availability, not data recovery.
- It doesn't cover correlated events the model ignores, such as a software bug, an operator mistake or a lost cloud account.

> [!NOTE]
> Exact durability and availability figures, and what the SLA pays, differ per provider, storage class and region, and they change. Read the provider's current documentation rather than relying on a number you remember.

Providers reach high durability by keeping data redundant across independent failure domains: disks, servers, racks and **availability zones** (separate data centres in one region). They use two main techniques.

**Replication** stores several full copies, for example three copies in three zones. It is simple, reads are fast (any copy can serve them) and a lost copy is simply re-copied. The cost is storage: three copies means 3x raw capacity.

**Erasure coding** splits an object into *k* data fragments and computes *m* parity fragments from them, usually with a Reed-Solomon code. Any *k* of the *k + m* fragments are enough to rebuild the object. With 4 data and 2 parity fragments (written "4+2") the system survives the loss of any two fragments while storing only 1.5x the data. The trade-offs are central processing unit (CPU) time for encoding, more network traffic to rebuild a lost fragment, and extra latency for small objects (many systems replicate small or hot data and erasure-code large or cold data).

<figure class="diagram" style="--dg-w: 1100px"><svg xmlns="http://www.w3.org/2000/svg" class="dg" viewBox="0 0 1100 420" width="1100" height="420" role="img" aria-labelledby="storage-replication-erasure-title storage-replication-erasure-desc" font-family="Inter, system-ui, -apple-system, Segoe UI, Roboto, sans-serif" xml:lang="en">
<title id="storage-replication-erasure-title">Replication compared with erasure coding</title>
<desc id="storage-replication-erasure-desc">Left: replication stores three full copies of a 12 MB object, one in each of zone A, zone B and zone C. The overhead is 3x, the data survives two lost copies, and it is simple with fast reads. Right: erasure coding with 4 data and 2 parity fragments splits the same object into six fragments of 3 MB. Two fragments, the second data fragment and the first parity fragment, are shown as lost, and the object is rebuilt from the remaining four. The overhead is 1.5x, and any 4 of the 6 fragments rebuild the object.</desc>
<rect class="dg-bg" x="0.5" y="0.5" width="1099" height="419" rx="16" fill="#ffffff" stroke="#e3e3dd"/>
<g class="dg-l1"><rect class="dg-tint" x="20" y="20" width="500" height="380" rx="16" fill="#f0f6f9" stroke="#9ac3d9" stroke-width="1.2" stroke-dasharray="6 5"/>
<g class="dg-icon" transform="translate(36 33) scale(0.7500)" fill="none" stroke="#0369a1" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M21 8a2 2 0 0 0-1-1.73l-7-4a2 2 0 0 0-2 0l-7 4A2 2 0 0 0 3 8v8a2 2 0 0 0 1 1.73l7 4a2 2 0 0 0 2 0l7-4A2 2 0 0 0 21 16Z"/><path d="m3.3 7 8.7 5 8.7-5"/><path d="M12 22V12"/></g>
<text x="62" y="47" font-size="13" class="dg-ink" fill="#0369a1" font-weight="700">Replication: 3 full copies</text></g>
<g class="dg-l4"><rect class="dg-tint" x="540" y="20" width="540" height="380" rx="16" fill="#fbf4f0" stroke="#e7b39e" stroke-width="1.2" stroke-dasharray="6 5"/>
<g class="dg-icon" transform="translate(556 33) scale(0.7500)" fill="none" stroke="#c2410c" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect width="7" height="7" x="3" y="3" rx="1"/><rect width="7" height="7" x="14" y="3" rx="1"/><rect width="7" height="7" x="14" y="14" rx="1"/><rect width="7" height="7" x="3" y="14" rx="1"/></g>
<text x="582" y="47" font-size="13" class="dg-ink" fill="#c2410c" font-weight="700">Erasure coding: 4 data + 2 parity fragments</text></g>
<rect class="dg-card" x="140" y="64" width="260" height="88" rx="12" fill="#ffffff" stroke="#cfcfc8"/>
<g class="dg-icon-neutral" transform="translate(156 80) scale(1.2500)" fill="none" stroke="#17171c" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><path d="M15 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V7Z"/><path d="M14 2v4a2 2 0 0 0 2 2h4"/><path d="M10 9H8"/><path d="M16 13H8"/><path d="M16 17H8"/></g>
<text x="198" y="101" font-size="15" class="dg-text" fill="#17171c" font-weight="700">Object</text>
<text x="158" y="132" font-size="13" class="dg-muted" fill="#5c5c68" font-weight="500">12 MB · to be stored durably</text>
<g class="dg-l1"><path class="dg-stroke" d="M270 154V196H110V244" stroke="#0369a1" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-l1"><path class="dg-stroke" d="M104 239l6 8 6-8" stroke="#0369a1" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<rect class="dg-card" x="40" y="252" width="140" height="96" rx="12" fill="#ffffff" stroke="#cfcfc8"/>
<g class="dg-l1"><g class="dg-icon" transform="translate(56 268) scale(1.2500)" fill="none" stroke="#0369a1" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><rect width="20" height="8" x="2" y="2" rx="2"/><rect width="20" height="8" x="2" y="14" rx="2"/><path d="M6 6h.01"/><path d="M6 18h.01"/></g></g>
<g class="dg-l1"><text x="98" y="289" font-size="15" class="dg-ink" fill="#0369a1" font-weight="700">Zone A</text></g>
<text x="58" y="320" font-size="13" class="dg-muted" fill="#5c5c68" font-weight="500">full copy</text>
<text x="58" y="340" font-size="13" class="dg-muted" fill="#5c5c68" font-weight="500">12 MB</text>
<g class="dg-l1"><path class="dg-stroke" d="M270 154V196H270V244" stroke="#0369a1" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-l1"><path class="dg-stroke" d="M264 239l6 8 6-8" stroke="#0369a1" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<rect class="dg-card" x="200" y="252" width="140" height="96" rx="12" fill="#ffffff" stroke="#cfcfc8"/>
<g class="dg-l1"><g class="dg-icon" transform="translate(216 268) scale(1.2500)" fill="none" stroke="#0369a1" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><rect width="20" height="8" x="2" y="2" rx="2"/><rect width="20" height="8" x="2" y="14" rx="2"/><path d="M6 6h.01"/><path d="M6 18h.01"/></g></g>
<g class="dg-l1"><text x="258" y="289" font-size="15" class="dg-ink" fill="#0369a1" font-weight="700">Zone B</text></g>
<text x="218" y="320" font-size="13" class="dg-muted" fill="#5c5c68" font-weight="500">full copy</text>
<text x="218" y="340" font-size="13" class="dg-muted" fill="#5c5c68" font-weight="500">12 MB</text>
<g class="dg-l1"><path class="dg-stroke" d="M270 154V196H430V244" stroke="#0369a1" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-l1"><path class="dg-stroke" d="M424 239l6 8 6-8" stroke="#0369a1" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<rect class="dg-card" x="360" y="252" width="140" height="96" rx="12" fill="#ffffff" stroke="#cfcfc8"/>
<g class="dg-l1"><g class="dg-icon" transform="translate(376 268) scale(1.2500)" fill="none" stroke="#0369a1" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><rect width="20" height="8" x="2" y="2" rx="2"/><rect width="20" height="8" x="2" y="14" rx="2"/><path d="M6 6h.01"/><path d="M6 18h.01"/></g></g>
<g class="dg-l1"><text x="418" y="289" font-size="15" class="dg-ink" fill="#0369a1" font-weight="700">Zone C</text></g>
<text x="378" y="320" font-size="13" class="dg-muted" fill="#5c5c68" font-weight="500">full copy</text>
<text x="378" y="340" font-size="13" class="dg-muted" fill="#5c5c68" font-weight="500">12 MB</text>
<g class="dg-l1"><rect class="dg-solid" x="40" y="362" width="434" height="26" rx="13" fill="#0369a1"/><text x="257" y="379.5" font-size="12" class="dg-on" fill="#ffffff" font-weight="700" text-anchor="middle">overhead 3x · survives 2 lost copies · simple, fast reads</text></g>
<rect class="dg-card" x="680" y="64" width="260" height="88" rx="12" fill="#ffffff" stroke="#cfcfc8"/>
<g class="dg-icon-neutral" transform="translate(696 80) scale(1.2500)" fill="none" stroke="#17171c" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><path d="M15 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V7Z"/><path d="M14 2v4a2 2 0 0 0 2 2h4"/><path d="M10 9H8"/><path d="M16 13H8"/><path d="M16 17H8"/></g>
<text x="738" y="101" font-size="15" class="dg-text" fill="#17171c" font-weight="700">Object</text>
<text x="698" y="132" font-size="13" class="dg-muted" fill="#5c5c68" font-weight="500">12 MB · to be stored durably</text>
<g class="dg-l4"><path class="dg-stroke" d="M810 154V196H592V244" stroke="#c2410c" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-l4"><path class="dg-stroke" d="M586 239l6 8 6-8" stroke="#c2410c" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-l1"><rect class="dg-tint" x="556" y="252" width="72" height="80" rx="10" fill="#e6f0f6" stroke="#68a5c7" stroke-width="1.4"/><text x="592" y="288" font-size="15" class="dg-ink" fill="#0369a1" font-weight="700" text-anchor="middle">D1</text><text x="592" y="310" font-size="12" class="dg-ink" fill="#0369a1" font-weight="600" text-anchor="middle">data</text></g>
<g class="dg-l4"><path class="dg-stroke" d="M810 154V196H676V244" stroke="#c2410c" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-l4"><path class="dg-stroke" d="M670 239l6 8 6-8" stroke="#c2410c" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-warn"><rect class="dg-tint" x="640" y="252" width="72" height="80" rx="10" fill="#f8e8e8" stroke="#d57777" stroke-width="1.4" stroke-dasharray="5 4"/><text x="676" y="288" font-size="15" class="dg-ink" fill="#b91c1c" font-weight="700" text-anchor="middle">D2</text><text x="676" y="310" font-size="12" class="dg-ink" fill="#b91c1c" font-weight="600" text-anchor="middle">lost</text></g>
<g class="dg-l4"><path class="dg-stroke" d="M810 154V196H760V244" stroke="#c2410c" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-l4"><path class="dg-stroke" d="M754 239l6 8 6-8" stroke="#c2410c" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-l1"><rect class="dg-tint" x="724" y="252" width="72" height="80" rx="10" fill="#e6f0f6" stroke="#68a5c7" stroke-width="1.4"/><text x="760" y="288" font-size="15" class="dg-ink" fill="#0369a1" font-weight="700" text-anchor="middle">D3</text><text x="760" y="310" font-size="12" class="dg-ink" fill="#0369a1" font-weight="600" text-anchor="middle">data</text></g>
<g class="dg-l4"><path class="dg-stroke" d="M810 154V196H844V244" stroke="#c2410c" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-l4"><path class="dg-stroke" d="M838 239l6 8 6-8" stroke="#c2410c" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-l1"><rect class="dg-tint" x="808" y="252" width="72" height="80" rx="10" fill="#e6f0f6" stroke="#68a5c7" stroke-width="1.4"/><text x="844" y="288" font-size="15" class="dg-ink" fill="#0369a1" font-weight="700" text-anchor="middle">D4</text><text x="844" y="310" font-size="12" class="dg-ink" fill="#0369a1" font-weight="600" text-anchor="middle">data</text></g>
<g class="dg-l4"><path class="dg-stroke" d="M810 154V196H928V244" stroke="#c2410c" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-l4"><path class="dg-stroke" d="M922 239l6 8 6-8" stroke="#c2410c" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-warn"><rect class="dg-tint" x="892" y="252" width="72" height="80" rx="10" fill="#f8e8e8" stroke="#d57777" stroke-width="1.4" stroke-dasharray="5 4"/><text x="928" y="288" font-size="15" class="dg-ink" fill="#b91c1c" font-weight="700" text-anchor="middle">P1</text><text x="928" y="310" font-size="12" class="dg-ink" fill="#b91c1c" font-weight="600" text-anchor="middle">lost</text></g>
<g class="dg-l4"><path class="dg-stroke" d="M810 154V196H1012V244" stroke="#c2410c" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-l4"><path class="dg-stroke" d="M1006 239l6 8 6-8" stroke="#c2410c" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-l4"><rect class="dg-tint" x="976" y="252" width="72" height="80" rx="10" fill="#f9ece7" stroke="#da8d6d" stroke-width="1.4"/><text x="1012" y="288" font-size="15" class="dg-ink" fill="#c2410c" font-weight="700" text-anchor="middle">P2</text><text x="1012" y="310" font-size="12" class="dg-ink" fill="#c2410c" font-weight="600" text-anchor="middle">parity</text></g>
<text x="560" y="354" font-size="12" class="dg-muted" fill="#5c5c68" font-weight="500">Each fragment is 3 MB; two lost fragments are rebuilt from the other four</text>
<g class="dg-l4"><rect class="dg-solid" x="560" y="362" width="420" height="26" rx="13" fill="#c2410c"/><text x="770" y="379.5" font-size="12" class="dg-on" fill="#ffffff" font-weight="700" text-anchor="middle">overhead 1.5x · any 4 of 6 fragments rebuild the object</text></g>
</svg></figure>

*Figure 2. Replication buys simplicity with capacity; erasure coding gives similar protection for half the overhead.*

Block volumes are usually replicated inside one zone, so a volume survives a disk failure but not the loss of a zone. For zone-level protection you copy data elsewhere (snapshots, replication to another region).

> [!CHECK]
> A provider advertises 99.999999999% durability for a bucket. You accidentally run a script that deletes every object in it. What does that figure tell you about getting the data back?
>
> - [ ] The provider will restore all objects, because the data can't be lost
> - [ ] The objects remain recoverable for as long as the bucket exists
> - [x] Nothing: durability is about hardware loss, and the service durably stores your delete too
> - [ ] Only the availability figure matters here
>
> Durability protects against disks, servers and zones failing. It does not undo deletions or overwrites made with valid credentials. Versioning and backups do.

### Storage classes, versioning and lifecycle rules
Not all data is read equally often. Last week's uploads are read daily; a seven-year-old audit log is almost never read. A **storage class** (or **tier**) lets you pay according to how often you access the data. Names differ by provider (Amazon S3 Standard, Standard-IA and Glacier; Azure Hot, Cool and Archive; Google Standard, Nearline, Coldline and Archive), but the model is the same:

| Tier | Storage cost | Retrieval | Typical use |
|---|---|---|---|
| Hot / standard | Highest | Free or very cheap, milliseconds | Active data, websites, application assets |
| Infrequent access / cool | Lower | Per-GB retrieval fee, still milliseconds | Backups read now and then, older media |
| Archive / cold | Lowest | Higher retrieval fee, minutes to hours (a restore request first) | Compliance records, long-term backups |

Colder tiers move cost from storing to reading: cheap per gigabyte per month, but with retrieval fees, a **minimum storage duration** (deleting early is still charged), sometimes a **minimum object size**, and slower first access. Archiving a dataset you read weekly can cost more than keeping it hot. Durability is usually the same across classes; availability and retrieval differ.

> [!NOTE]
> Prices, minimum durations, minimum sizes and retrieval times change and differ per provider, so this lesson gives none. Always check the current price list and model your own access pattern before moving data to a colder tier.

A **lifecycle rule** automates this. You write the policy once and the service applies it: "under `tmp/`, delete objects 30 days after creation"; "move objects to the cool tier after 60 days and to archive after 365"; "delete noncurrent versions after 90 days". Rules match objects by prefix, tag or size. They run in the background, usually once a day, so an object may live somewhat longer than the exact time you set.

**Versioning** protects against overwrites and deletes. When you enable it on a bucket, every `PUT` to an existing key creates a new **version** with its own version ID instead of replacing the object, and the old versions stay. A plain `DELETE` doesn't remove data: it adds a **delete marker**, a zero-byte placeholder that becomes the current version. Because the current version is a delete marker, `GET` returns "not found" and the key disappears from normal listings, but earlier versions are intact. To undo, you delete the marker or copy an older version back so it becomes current. To really erase data you delete a specific version ID.

<figure class="diagram" style="--dg-w: 1100px"><svg xmlns="http://www.w3.org/2000/svg" class="dg" viewBox="0 0 1100 460" width="1100" height="460" role="img" aria-labelledby="storage-versioning-title storage-versioning-desc" font-family="Inter, system-ui, -apple-system, Segoe UI, Roboto, sans-serif" xml:lang="en">
<title id="storage-versioning-title">Versioning with a delete marker</title>
<desc id="storage-versioning-desc">Four steps on a bucket with versioning enabled. 1. PUT notes.txt creates version v1. 2. PUT notes.txt again creates version v2. 3. DELETE notes.txt adds a delete marker, which becomes the current version. 4. Copying v1 back creates version v4, a copy of v1. After step 3 the bucket holds the delete marker (current), v2 and v1, and GET notes.txt returns 404 Not Found although v1 and v2 still exist. After step 4 it holds v4 (current), the delete marker, v2 and v1, and GET notes.txt returns the content of v1.</desc>
<rect class="dg-bg" x="0.5" y="0.5" width="1099" height="459" rx="16" fill="#ffffff" stroke="#e3e3dd"/>
<rect class="dg-card" x="24" y="24" width="237.5" height="118" rx="12" fill="#ffffff" stroke="#cfcfc8"/>
<g class="dg-l4"><circle class="dg-solid" cx="46" cy="48" r="12" fill="#c2410c"/><text x="46" y="52.5" font-size="12.5" class="dg-on" fill="#ffffff" font-weight="700" text-anchor="middle">1</text></g>
<g class="dg-l4"><g class="dg-icon" transform="translate(221.5 36) scale(1.0833)" fill="none" stroke="#c2410c" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><path d="M15 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V7Z"/><path d="M14 2v4a2 2 0 0 0 2 2h4"/><path d="M10 9H8"/><path d="M16 13H8"/><path d="M16 17H8"/></g></g>
<text x="38" y="86" font-size="14" class="dg-text" fill="#17171c" font-weight="700">PUT notes.txt</text>
<text x="38" y="106" font-size="12" class="dg-muted" fill="#5c5c68" font-weight="500">creates version v1</text>
<path class="dg-line" d="M267.5 83H289.5" stroke="#cfcfc8" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/>
<path class="dg-line" d="M281.5 77l8 6-8 6" stroke="#cfcfc8" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/>
<rect class="dg-card" x="295.5" y="24" width="237.5" height="118" rx="12" fill="#ffffff" stroke="#cfcfc8"/>
<g class="dg-l4"><circle class="dg-solid" cx="317.5" cy="48" r="12" fill="#c2410c"/><text x="317.5" y="52.5" font-size="12.5" class="dg-on" fill="#ffffff" font-weight="700" text-anchor="middle">2</text></g>
<g class="dg-l4"><g class="dg-icon" transform="translate(493 36) scale(1.0833)" fill="none" stroke="#c2410c" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><path d="M15 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V7Z"/><path d="M14 2v4a2 2 0 0 0 2 2h4"/><path d="M10 9H8"/><path d="M16 13H8"/><path d="M16 17H8"/></g></g>
<text x="309.5" y="86" font-size="14" class="dg-text" fill="#17171c" font-weight="700">PUT notes.txt again</text>
<text x="309.5" y="106" font-size="12" class="dg-muted" fill="#5c5c68" font-weight="500">creates version v2</text>
<path class="dg-line" d="M539 83H561" stroke="#cfcfc8" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/>
<path class="dg-line" d="M553 77l8 6-8 6" stroke="#cfcfc8" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/>
<rect class="dg-card" x="567" y="24" width="237.5" height="118" rx="12" fill="#ffffff" stroke="#cfcfc8"/>
<g class="dg-l4"><circle class="dg-solid" cx="589" cy="48" r="12" fill="#c2410c"/><text x="589" y="52.5" font-size="12.5" class="dg-on" fill="#ffffff" font-weight="700" text-anchor="middle">3</text></g>
<g class="dg-l4"><g class="dg-icon" transform="translate(764.5 36) scale(1.0833)" fill="none" stroke="#c2410c" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><path d="m21.73 18-8-14a2 2 0 0 0-3.48 0l-8 14A2 2 0 0 0 4 21h16a2 2 0 0 0 1.73-3"/><path d="M12 9v4"/><path d="M12 17h.01"/></g></g>
<text x="581" y="86" font-size="14" class="dg-text" fill="#17171c" font-weight="700">DELETE notes.txt</text>
<text x="581" y="106" font-size="12" class="dg-muted" fill="#5c5c68" font-weight="500">adds a delete marker</text>
<path class="dg-line" d="M810.5 83H832.5" stroke="#cfcfc8" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/>
<path class="dg-line" d="M824.5 77l8 6-8 6" stroke="#cfcfc8" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/>
<rect class="dg-card" x="838.5" y="24" width="237.5" height="118" rx="12" fill="#ffffff" stroke="#cfcfc8"/>
<g class="dg-l4"><circle class="dg-solid" cx="860.5" cy="48" r="12" fill="#c2410c"/><text x="860.5" y="52.5" font-size="12.5" class="dg-on" fill="#ffffff" font-weight="700" text-anchor="middle">4</text></g>
<g class="dg-l4"><g class="dg-icon" transform="translate(1036 36) scale(1.0833)" fill="none" stroke="#c2410c" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><path d="M21 8a2 2 0 0 0-1-1.73l-7-4a2 2 0 0 0-2 0l-7 4A2 2 0 0 0 3 8v8a2 2 0 0 0 1 1.73l7 4a2 2 0 0 0 2 0l7-4A2 2 0 0 0 21 16Z"/><path d="m3.3 7 8.7 5 8.7-5"/><path d="M12 22V12"/></g></g>
<text x="852.5" y="86" font-size="14" class="dg-text" fill="#17171c" font-weight="700">Copy v1 back</text>
<text x="852.5" y="106" font-size="12" class="dg-muted" fill="#5c5c68" font-weight="500">creates version v4,</text>
<text x="852.5" y="123" font-size="12" class="dg-muted" fill="#5c5c68" font-weight="500">a copy of v1</text>
<rect class="dg-panel" x="24" y="170" width="506" height="226" rx="12" fill="#f0f0ec"/>
<text x="44" y="198" font-size="11" class="dg-muted" fill="#5c5c68" font-weight="700" letter-spacing="1.2">AFTER STEP 3</text>
<text x="510" y="198" font-size="12.5" class="dg-muted" fill="#5c5c68" font-weight="600" text-anchor="end" font-family="JetBrains Mono, ui-monospace, Menlo, Consolas, monospace">notes.txt</text>
<g class="dg-warn"><rect class="dg-solid" x="44" y="214" width="466" height="32" rx="10" fill="#b91c1c" stroke="#d57777" stroke-width="1.2"/><text x="60" y="235" font-size="13" class="dg-on" fill="#ffffff" font-weight="700">delete marker (current)</text></g>
<g class="dg-l4"><rect class="dg-tint" x="44" y="254" width="466" height="32" rx="10" fill="#f9ece7" stroke="#da8d6d" stroke-width="1.2"/><text x="60" y="275" font-size="13" class="dg-ink" fill="#c2410c" font-weight="700">version v2</text></g>
<g class="dg-l4"><rect class="dg-tint" x="44" y="294" width="466" height="32" rx="10" fill="#f9ece7" stroke="#da8d6d" stroke-width="1.2"/><text x="60" y="315" font-size="13" class="dg-ink" fill="#c2410c" font-weight="700">version v1</text></g>
<g class="dg-warn"><text x="44" y="362" font-size="12.5" class="dg-ink" fill="#b91c1c" font-weight="700">GET notes.txt → 404 Not Found, but v1 and v2 still exist</text></g>
<rect class="dg-panel" x="570" y="170" width="506" height="266" rx="12" fill="#f0f0ec"/>
<text x="590" y="198" font-size="11" class="dg-muted" fill="#5c5c68" font-weight="700" letter-spacing="1.2">AFTER STEP 4</text>
<text x="1056" y="198" font-size="12.5" class="dg-muted" fill="#5c5c68" font-weight="600" text-anchor="end" font-family="JetBrains Mono, ui-monospace, Menlo, Consolas, monospace">notes.txt</text>
<g class="dg-l4"><rect class="dg-solid" x="590" y="214" width="466" height="32" rx="10" fill="#c2410c" stroke="#da8d6d" stroke-width="1.2"/><text x="606" y="235" font-size="13" class="dg-on" fill="#ffffff" font-weight="700">version v4 (current)</text></g>
<g class="dg-warn"><rect class="dg-tint" x="590" y="254" width="466" height="32" rx="10" fill="#f8e8e8" stroke="#d57777" stroke-width="1.2"/><text x="606" y="275" font-size="13" class="dg-ink" fill="#b91c1c" font-weight="700">delete marker</text></g>
<g class="dg-l4"><rect class="dg-tint" x="590" y="294" width="466" height="32" rx="10" fill="#f9ece7" stroke="#da8d6d" stroke-width="1.2"/><text x="606" y="315" font-size="13" class="dg-ink" fill="#c2410c" font-weight="700">version v2</text></g>
<g class="dg-l4"><rect class="dg-tint" x="590" y="334" width="466" height="32" rx="10" fill="#f9ece7" stroke="#da8d6d" stroke-width="1.2"/><text x="606" y="355" font-size="13" class="dg-ink" fill="#c2410c" font-weight="700">version v1</text></g>
<g class="dg-ok"><text x="590" y="402" font-size="12.5" class="dg-ink" fill="#15803d" font-weight="700">GET notes.txt → the content of v1; nothing was lost</text></g>
</svg></figure>

*Figure 3. With versioning, a delete only hides the object behind a marker. Restoring means making an older version current again.*

Versioning has a cost: every version is stored and billed. Pair it with a lifecycle rule that expires noncurrent versions after a set number of days, or the bucket grows forever. On Amazon S3, versioning can't be switched off once enabled, only suspended; check your provider.

> [!CHECK]
> Versioning is enabled. You upload `notes.txt` twice and then delete it with a normal delete request. What does the bucket contain?
>
> - [ ] Nothing: a delete removes all versions
> - [ ] Only the second version, because the first was overwritten
> - [x] Both versions plus a delete marker, which makes the key look deleted
> - [ ] A single copy of the file in the archive tier
>
> A delete on a versioned bucket adds a delete marker. Normal reads see "not found", but both versions remain until you delete them by version ID or a lifecycle rule expires them.

### Sharing objects and controlling access
A new bucket is **private**: only the account that owns it, and principals it has granted access to, can read it. (Identities, roles and policies are the subject of [module 07](../07-cloud-iam/README.md).) There are three ways to let others in, from safest to riskiest.

A **pre-signed URL** gives time-limited access to one object without sharing credentials. Your application uses its secret key to compute a signature over the HTTP method, the object path and an expiry time, and appends it to the URL. Anyone who holds the URL can perform that one operation (usually `GET` or `PUT`) until it expires. Signing happens locally, with no call to the service. The service checks the signature, the expiry and that the signer is still allowed to read the object. Typical uses: a download link in an email, or letting a browser upload a photo straight to the bucket instead of through your server. Keep expiries short, treat the URL as a secret (it appears in logs and browser history), and remember it can stop working earlier if the signing credentials are temporary and expire first.

<figure class="diagram" style="--dg-w: 1100px"><svg xmlns="http://www.w3.org/2000/svg" class="dg" viewBox="0 0 1100 473" width="1100" height="473" role="img" aria-labelledby="storage-presigned-url-title storage-presigned-url-desc" font-family="Inter, system-ui, -apple-system, Segoe UI, Roboto, sans-serif" xml:lang="en">
<title id="storage-presigned-url-title">How a pre-signed URL works</title>
<desc id="storage-presigned-url-desc">Three parties: the application that holds the credentials, the object storage service whose bucket stays private, and a recipient without credentials. 1. The application signs locally with its secret key, over the method, the path and the expiry, producing a signature; no request is sent to the service. 2. It shares the URL, valid for 5 minutes, with the recipient. 3. The recipient sends GET for the object with the signature in the query string. The service recomputes the signature, checks the expiry and checks that the signer is allowed to read the object. 4. It answers 200 OK with the object bytes, or 403 Forbidden if the URL expired or was altered.</desc>
<rect class="dg-bg" x="0.5" y="0.5" width="1099" height="472" rx="16" fill="#ffffff" stroke="#e3e3dd"/>
<path class="dg-line" d="M130 98V457" stroke="#cfcfc8" stroke-width="1.5" fill="none" stroke-linecap="round" stroke-linejoin="round" stroke-dasharray="7 6"/>
<path class="dg-line" d="M550 98V457" stroke="#cfcfc8" stroke-width="1.5" fill="none" stroke-linecap="round" stroke-linejoin="round" stroke-dasharray="7 6"/>
<path class="dg-line" d="M970 98V457" stroke="#cfcfc8" stroke-width="1.5" fill="none" stroke-linecap="round" stroke-linejoin="round" stroke-dasharray="7 6"/>
<rect class="dg-card" x="25" y="24" width="210" height="74" rx="12" fill="#ffffff" stroke="#cfcfc8"/>
<g class="dg-l7"><g class="dg-icon" transform="translate(39 38) scale(1.1667)" fill="none" stroke="#6d28d9" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><path d="M20 16V7a2 2 0 0 0-2-2H6a2 2 0 0 0-2 2v9m16 0H4m16 0 1.28 2.55a1 1 0 0 1-.9 1.45H3.62a1 1 0 0 1-.9-1.45L4 16"/></g></g>
<text x="77" y="56" font-size="14" class="dg-text" fill="#17171c" font-weight="700">Application</text>
<text x="77" y="76" font-size="12" class="dg-muted" fill="#5c5c68" font-weight="500">has the credentials</text>
<rect class="dg-card" x="445" y="24" width="210" height="74" rx="12" fill="#ffffff" stroke="#cfcfc8"/>
<g class="dg-l4"><g class="dg-icon" transform="translate(459 38) scale(1.1667)" fill="none" stroke="#c2410c" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><path d="M17.5 19H9a7 7 0 1 1 6.71-9h1.79a4.5 4.5 0 1 1 0 9Z"/></g></g>
<text x="497" y="56" font-size="14" class="dg-text" fill="#17171c" font-weight="700">Object storage</text>
<text x="497" y="76" font-size="12" class="dg-muted" fill="#5c5c68" font-weight="500">bucket stays private</text>
<rect class="dg-card" x="865" y="24" width="210" height="74" rx="12" fill="#ffffff" stroke="#cfcfc8"/>
<g class="dg-icon-neutral" transform="translate(879 38) scale(1.1667)" fill="none" stroke="#17171c" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><rect width="10" height="14" x="3" y="8" rx="2"/><path d="M5 4a2 2 0 0 1 2-2h12a2 2 0 0 1 2 2v12a2 2 0 0 1-2 2h-2.4"/><path d="M8 18h.01"/></g>
<text x="917" y="56" font-size="14" class="dg-text" fill="#17171c" font-weight="700">Recipient</text>
<text x="917" y="76" font-size="12" class="dg-muted" fill="#5c5c68" font-weight="500">no credentials</text>
<g class="dg-l7"><rect class="dg-tint" x="14" y="112" width="291" height="65" rx="8" fill="#f0eafb" stroke="#bd9eee" stroke-width="1.2"/>
<text x="159.5" y="129" font-size="12.5" class="dg-ink" fill="#6d28d9" font-weight="600" text-anchor="middle">1. Sign locally with your secret key:</text>
<text x="159.5" y="146" font-size="12.5" class="dg-ink" fill="#6d28d9" font-weight="600" text-anchor="middle">method + path + expiry → signature</text>
<text x="159.5" y="163" font-size="12.5" class="dg-ink" fill="#6d28d9" font-weight="600" text-anchor="middle">(no request is sent to the service)</text>
</g>
<text x="550" y="211" font-size="12.5" class="dg-text" fill="#17171c" font-weight="600" text-anchor="middle" font-family="JetBrains Mono, ui-monospace, Menlo, Consolas, monospace">Share the URL (5 minutes)</text>
<text x="550" y="228" font-size="12.5" class="dg-text" fill="#17171c" font-weight="500" text-anchor="middle" font-family="JetBrains Mono, ui-monospace, Menlo, Consolas, monospace">…/q3.txt?X-Amz-Expires=300&amp;X-Amz-Signature=…</text>
<g class="dg-l7"><circle class="dg-solid" cx="436.688" cy="207" r="11" fill="#6d28d9"/><text x="436.688" y="211.5" font-size="12" class="dg-on" fill="#ffffff" font-weight="700" text-anchor="middle">2</text></g>
<g class="dg-l7"><path class="dg-stroke" d="M134 242H964" stroke="#6d28d9" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-l7"><path class="dg-stroke" d="M956 236l8 6-8 6" stroke="#6d28d9" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<text x="760" y="280" font-size="12.5" class="dg-text" fill="#17171c" font-weight="600" text-anchor="middle" font-family="JetBrains Mono, ui-monospace, Menlo, Consolas, monospace">GET /reports/q3.txt?X-Amz-Signature=…</text>
<g class="dg-l4"><circle class="dg-solid" cx="600.938" cy="276" r="11" fill="#c2410c"/><text x="600.938" y="280.5" font-size="12" class="dg-on" fill="#ffffff" font-weight="700" text-anchor="middle">3</text></g>
<g class="dg-l4"><path class="dg-stroke" d="M966 294H556" stroke="#c2410c" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-l4"><path class="dg-stroke" d="M564 288l-8 6 8 6" stroke="#c2410c" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
<g class="dg-l4"><rect class="dg-tint" x="373" y="316" width="354" height="48" rx="8" fill="#f9ece7" stroke="#e4aa92" stroke-width="1.2"/>
<text x="550" y="333" font-size="12.5" class="dg-ink" fill="#c2410c" font-weight="600" text-anchor="middle">Recompute the signature, check the expiry,</text>
<text x="550" y="350" font-size="12.5" class="dg-ink" fill="#c2410c" font-weight="600" text-anchor="middle">check the signer is allowed to read the object</text>
</g>
<text x="760" y="398" font-size="12.5" class="dg-text" fill="#17171c" font-weight="600" text-anchor="middle" font-family="JetBrains Mono, ui-monospace, Menlo, Consolas, monospace">200 OK + object bytes</text>
<text x="760" y="415" font-size="12.5" class="dg-text" fill="#17171c" font-weight="500" text-anchor="middle" font-family="JetBrains Mono, ui-monospace, Menlo, Consolas, monospace">(403 Forbidden if expired or altered)</text>
<g class="dg-ok"><circle class="dg-solid" cx="661.938" cy="394" r="11" fill="#15803d"/><text x="661.938" y="398.5" font-size="12" class="dg-on" fill="#ffffff" font-weight="700" text-anchor="middle">4</text></g>
<g class="dg-ok"><path class="dg-stroke" d="M554 429H964" stroke="#15803d" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round" stroke-dasharray="7 6"/></g>
<g class="dg-ok"><path class="dg-stroke" d="M956 423l8 6-8 6" stroke="#15803d" stroke-width="2.5" fill="none" stroke-linecap="round" stroke-linejoin="round"/></g>
</svg></figure>

*Figure 4. A pre-signed URL is a capability: whoever holds it can use it until it expires, and the bucket never has to become public.*

A **bucket policy** is a JavaScript Object Notation (JSON) document attached to the bucket that says who may do what to which objects: the principals, the actions (such as `s3:GetObject`), the resources and optional conditions. Use it for durable rules, for example "this other account can read `reports/*`" or "deny any request that isn't over HTTPS". It is a resource-based policy; identity-based policies attached to users and roles work alongside it.

A **public bucket** (or public object) is readable by anyone on the Internet, with no signature, because a policy or access control list (ACL) grants access to the principal `*` ("everyone"). It is the most common cause of cloud data leaks: a policy meant for a website's images, applied too broadly, exposes customer exports; automated scanners look for open buckets all the time. To prevent it, providers offer **block public access**, a switch at bucket level (and often account or organisation level) that overrides any policy or ACL that would make data public. Leave it on everywhere, and turn it off only for a bucket that is deliberately a public website, ideally behind a content delivery network (CDN) rather than serving the bucket directly.

> [!CHECK]
> A partner needs to download one report from your private bucket this afternoon. They have no account with you. What is the safest option?
>
> - [ ] Make the whole bucket public and send the link
> - [ ] Share your access key and secret key so they can use the CLI
> - [x] Generate a pre-signed URL for that one object with a short expiry
> - [ ] Add a bucket policy that allows the principal `*` to read everything
>
> A pre-signed URL grants one operation on one object for a limited time, using your signature but never revealing your keys, and the bucket stays private.

### Snapshots, backups and encryption
A **snapshot** is a point-in-time copy of a volume (or file share), taken without detaching it. Snapshots are usually incremental: after the first, only the blocks that changed are stored, so they are fast and cheap. They are the quick way to roll back after a bad upgrade or to clone a disk. By default, however, a snapshot lives in the same provider, account and region as the volume. A snapshot is **crash-consistent** (like pulling the power cord) unless you quiesce the application, for example by flushing the database, first.

A **backup** is an independent copy that survives the loss of the original *and* of the account, region or system that held it. The difference is the failure domain: if the same mistake, ransomware or account compromise can delete both the data and its copies, you have a copy, not a backup. The classic guide is the **3-2-1 rule**:

- **3** copies of your data (the original plus two backups),
- on **2** different types of storage or services,
- with **1** copy off-site (another region, another provider or offline).

Modern practice adds an immutable copy (object-lock or write-once-read-many, "WORM", storage that can't be changed or deleted before a retention date) and, above all, **restore tests**: a backup you haven't restored is a hope, not a backup. Snapshots are a good first layer, and the 3-2-1 rule asks for more.

**Encryption at rest** protects stored data: if someone obtains the disks or a copy of the storage, they get ciphertext. Cloud storage can encrypt every object and volume with a symmetric key (typically Advanced Encryption Standard, AES, with 256 bits). What differs is who holds the key:

| Key model | Who manages the key | Notes |
|---|---|---|
| Provider-managed keys | The provider, transparently | Often on by default; no setup; you can't control or audit the key's use |
| Customer-managed keys | You, in the provider's **key management service (KMS)** | You set the key policy, rotate it and see every use in the audit log; disabling or deleting the key makes the data unreadable |
| Client-side or customer-provided keys | You, outside the provider | The provider never sees the plaintext key or data; strongest control, but you must keep the key safe or lose the data |

Services use **envelope encryption**: a unique data key encrypts each object, and the KMS key encrypts ("wraps") the data key. Only the small wrapped key goes to the KMS, which keeps it fast and keeps the master key inside it. Encryption at rest does not stop an authorised user from reading data: the service decrypts it for anyone the policy allows, so it protects against stolen media and some insider risks, not against misconfigured permissions.

**Encryption in transit** protects data on the network. Use **Transport Layer Security (TLS)**, the protocol behind HTTPS (TLS 1.3 is RFC 8446), for every call to a storage API, and mount network file shares with encryption enabled (NFS over TLS, SMB encryption). Without it, anyone on the path can read or alter the data. Most providers can enforce it with a bucket policy that rejects plain HTTP.

> [!CHECK]
> Your volume has a snapshot taken daily, stored in the same account and region. An attacker with stolen admin credentials deletes the volume. Why is the snapshot not a sufficient backup?
>
> - [ ] Snapshots can't be restored to new volumes
> - [ ] Snapshots are always unencrypted
> - [x] The same credentials can delete the snapshots too: the copy shares the failure domain
> - [ ] Snapshots are only stored for 24 hours
>
> A backup must survive the loss of the account or region that holds the original. The 3-2-1 rule asks for at least one copy elsewhere, ideally immutable.

## How it works
Follow one upload and one download of a private object, so the pieces fit together.

1. **The client signs the request.** The software development kit (SDK) or command-line interface (CLI) builds the HTTP request, hashes its headers and body, and signs them with your secret key (AWS Signature Version 4 is the usual scheme). The secret key is never sent; the signature and your access key ID are.
2. **The service authenticates and authorises.** It finds your secret from the access key ID, recomputes the signature, then evaluates the identity-based policies, the bucket policy and block public access to decide whether this action is allowed.
3. **The data is placed and protected.** The service splits the object, writes replicas or erasure-coded fragments to disks in several zones, and stores the metadata in an index. If versioning is on, it records a new version ID.
4. **The service answers.** Only after enough fragments are safely stored does it return `200 OK` and an **ETag** (an identifier of the content, often its Message-Digest 5, MD5, hash for simple uploads), so a successful upload is already durable.
5. **A download reverses the path.** The service checks the signature (or a pre-signed URL's query signature) and permissions, reads enough fragments to rebuild the object, and streams it back over TLS. For an archive-tier object, the first answer is an error until you request a restore.

Meanwhile, background jobs apply lifecycle rules, move objects between tiers, expire old versions and repair lost fragments.

> [!CHECK]
> In the upload flow, when does the service tell the client `200 OK`?
>
> - [ ] Immediately when the first byte arrives, then it stores the rest later
> - [x] After the object has been stored redundantly, so that success already means it is durable
> - [ ] Once the daily lifecycle job has processed it
> - [ ] Only when a second client has read it
>
> Object stores acknowledge a write only after enough copies or fragments are safely stored. Lifecycle jobs and tier moves happen later, in the background.

## Worked example
You design storage for the photo-sharing app from the start of the lesson. It has 2 TB of photos growing 100 GB a month, a PostgreSQL database and a shared folder of configuration templates.

| Data | Storage | Class and protection | Why |
|---|---|---|---|
| Photos and thumbnails | Object storage bucket `photos-prod` | Standard for 30 days, then infrequent access via a lifecycle rule; versioning on; noncurrent versions expire after 90 days | Read-heavy, huge, no in-place edits. Users upload through pre-signed `PUT` URLs; downloads go through a CDN |
| PostgreSQL data | Block volume on the database instance | Daily snapshot; copy to another region weekly | Needs low latency and IOPS; a single writer |
| Configuration templates | File share | Snapshot daily | Several app servers mount it and edit files in place |
| Backups | Separate bucket in a different account and region | Archive tier, object lock for 30 days | Satisfies 3-2-1: survives deletion of the main account |

Check the numbers. Keeping 2 TB of photos as three replicas needs 6 TB of raw disk (3x). A 4+2 erasure code needs 3 TB (1.5x), so for the same protection you store half as much. If the object store's availability target is 99.9%, plan for up to about 8.8 hours a year in which uploads may fail: the app should retry and show a friendly message, not lose the photo. Durability of the photos is a separate matter: an engineer deleting the prefix is covered by versioning, and the loss of the whole account by the cross-account backup.

Encryption: the bucket uses a customer-managed key in the KMS so that the security team controls the key policy and sees every use in the audit log; the database volume uses the provider-managed default. All traffic uses HTTPS, and block public access is on for the account.

## Common pitfalls
- **Treating durability as a backup** — eleven nines protects against hardware loss, not against your own `DELETE`. Enable versioning and keep an independent copy.
- **Public by accident** — one broad `Principal: "*"` policy or ACL exposes data. Keep block public access on and review policies.
- **Expecting folders** — in object storage, "folders" are prefixes. Renaming one rewrites every object under it, and an empty folder doesn't exist.
- **Versioning without lifecycle** — every overwrite is billed forever until a rule expires noncurrent versions.
- **Archiving data you still read** — retrieval fees and delays can cost more than staying in a hot tier. Model access patterns first.
- **Mounting one block volume on two servers** — an ordinary filesystem such as ext4 isn't built for concurrent writers and will corrupt. Use file storage, or a cluster filesystem.
- **Never testing restores** — backups fail silently. Restore on a schedule.
- **Long-lived pre-signed URLs** — a URL valid for days is a password in a log file. Use minutes, not days.

## Security considerations
- **Least privilege and no public access.** Grant the narrowest actions on the narrowest prefixes, never put access keys in code or repositories, and use short-lived credentials from roles where possible. Enable block public access at the account level.
- **Monitor and audit.** Turn on access logging and the provider's audit trail for the bucket and for KMS key use. Alert on changes to bucket policies and on new public access.
- **Protect against ransomware and mistakes.** Enable versioning, require multi-factor authentication (MFA) for permanent deletes where offered, use object lock for backups, and keep a copy in a separate account.
- **Encrypt everywhere.** Use TLS for all traffic and encryption at rest. For sensitive data, use customer-managed keys, restrict who can use or delete them, and remember that deleting a key permanently destroys the data it protects (called crypto-shredding, useful when you must erase data, dangerous by accident).
- **Treat pre-signed URLs as secrets.** Short expiry, one object, one operation; don't log or share them widely.
- **Plan for the shared-responsibility model.** The provider secures the service; you own the configuration, the keys, the permissions and the backups. Most storage incidents are misconfiguration, not provider failure.
- **Follow a standard.** NIST SP 800-209 gives security guidelines for storage infrastructure.

## Summary
- Block storage is a raw volume for one instance (low latency, IOPS); file storage is a shared filesystem (NFS or SMB); object storage holds whole objects in a flat bucket and is reached over HTTP, with the S3 API as the de facto standard.
- Durability means data isn't lost; availability means it can be reached. Replication copies whole objects; erasure coding splits them into data and parity fragments for less overhead. Eleven nines doesn't protect against deletion.
- Storage classes trade cheaper storage for retrieval fees and delay. Lifecycle rules move or expire data automatically. Versioning turns deletes into delete markers and keeps old versions.
- Share with a pre-signed URL, not by making the bucket public; keep block public access on.
- A snapshot is a quick point-in-time copy; a backup is an independent copy that follows the 3-2-1 rule and has been restore-tested.
- Encrypt at rest (provider-managed or customer-managed keys in a KMS) and in transit (TLS). Key ownership decides who controls access to the data.

## Glossary
| Term | Definition |
|---|---|
| Block storage | Storage presented as a raw volume of fixed-size blocks that a server formats with a filesystem. |
| Volume | A block storage device of a fixed size, attached to an instance. |
| Filesystem | The structure (such as ext4, XFS or NTFS) that organises blocks into files and directories. |
| IOPS | Input/output operations per second: how many reads and writes a volume handles. |
| Throughput | Bytes transferred per second; IOPS multiplied by the size of each operation. |
| Latency | The time one storage operation takes to complete. |
| File storage | A shared filesystem served over the network, with directories, permissions and locking. |
| Network File System (NFS) | A protocol for mounting remote file shares, common on Linux and Unix (NFSv4.1: RFC 8881). |
| Server Message Block (SMB) | A file-sharing protocol, common on Windows. |
| POSIX semantics | File behaviour defined by the Portable Operating System Interface: partial updates, atomic rename, permissions and locks. |
| Object storage | Storage of whole objects with metadata in flat buckets, accessed through an HTTP API. |
| Object | The data stored in a bucket, with its key and metadata. |
| Bucket | A named container for objects in object storage. |
| Key | The unique name of an object within a bucket. |
| Metadata | Information about an object: system fields (size, content type, date) and custom key-value pairs. |
| Flat namespace | A naming scheme with no directory hierarchy: keys are single strings. |
| Prefix | The leading part of a key, usually ending in `/`, used to group and list objects like folders. |
| Multipart upload | Uploading a large object in parts that can be sent and retried independently. |
| S3 API | The HTTP interface of Amazon S3, implemented by many providers and open-source servers. |
| Strong read-after-write consistency | A guarantee that after a successful write, every later read returns the new data. |
| Durability | The probability that stored data is not lost or corrupted over time. |
| Availability | The fraction of time a service can be reached and answers requests. |
| Eleven nines | A durability figure of 99.999999999%, usually per object per year; a design target, not protection from deletion. |
| Service-level agreement (SLA) | The provider's commitment, usually about availability, and the credits owed if it is missed. |
| Availability zone | A physically separate data centre, or group of them, within a cloud region. |
| Replication | Redundancy by storing several full copies of the data. |
| Erasure coding | Redundancy by splitting data into k data fragments plus m parity fragments, where any k rebuild the data. |
| Storage class (tier) | A price and performance level, such as hot, infrequent access or archive. |
| Minimum storage duration | The shortest time you are billed for an object in a colder tier, even if you delete it earlier. |
| Lifecycle rule | An automatic policy that moves or deletes objects by age, prefix or tag. |
| Versioning | A bucket feature that keeps every version of an object instead of overwriting it. |
| Version | One stored state of an object, identified by a version ID. |
| Delete marker | A placeholder that becomes the current version when you delete a versioned object; older versions remain. |
| Pre-signed URL | A URL with an embedded signature and expiry that grants one operation on one object. |
| Bucket policy | A JSON resource policy attached to a bucket that allows or denies actions to principals. |
| Public bucket | A bucket whose data anyone on the Internet can read without credentials. |
| Block public access | A setting that overrides any policy or ACL that would make a bucket or its objects public. |
| Access control list (ACL) | An older, per-object or per-bucket permission list; prefer policies. |
| Content delivery network (CDN) | A network of edge servers that caches and serves content close to users. |
| Snapshot | A point-in-time, usually incremental copy of a volume or share, kept by default alongside the original. |
| Crash-consistent | A copy equivalent to the disk at the moment of a power loss, without flushing the application. |
| Backup | An independent copy that survives the loss of the original and of its account, region or system. |
| 3-2-1 rule | Keep 3 copies on 2 types of storage, with 1 copy off-site. |
| Object lock / WORM | Write once, read many: prevents an object being changed or deleted until a retention date. |
| Encryption at rest | Encrypting stored data so that stolen media or copies are unreadable. |
| Key management service (KMS) | A service that creates, stores, controls and audits the use of encryption keys. |
| Provider-managed key | An encryption key created and controlled by the cloud provider. |
| Customer-managed key | An encryption key you control in a KMS, with your own policy, rotation and audit trail. |
| Envelope encryption | Encrypting data with a data key, then encrypting that data key with a KMS key. |
| Crypto-shredding | Making data unrecoverable by destroying the key that encrypts it. |
| Transport Layer Security (TLS) | The protocol that encrypts and authenticates network connections, such as HTTPS (TLS 1.3: RFC 8446). |
| ETag | An identifier of an object's content returned by the service, often an MD5 hash for simple uploads. |
| Multi-factor authentication (MFA) | Requiring a second proof of identity besides a password or key. |

## Check your understanding
1. A colleague says, "Our bucket has eleven nines of durability, so we don't need backups." Explain two ways data can still be lost.
2. You must store 500 TB of research data that is read once a year. Compare a hot tier with an archive tier. What would you measure before deciding?
3. Why does `aws s3 rm` on a versioned bucket not free any storage, and how would you remove the data for good?
4. Explain how a pre-signed URL lets a browser upload a photo directly to a private bucket. Who holds the credentials, and what limits the damage if the URL leaks?
5. Your team stores snapshots of a database volume in the same account. Redesign this to meet the 3-2-1 rule, and say how you would prove it works.
6. A compliance officer asks who can read objects encrypted with provider-managed keys and who can read those encrypted with customer-managed keys. What do you answer?

## References
- [NIST SP 800-209](https://csrc.nist.gov/pubs/sp/800/209/final) — Security Guidelines for Storage Infrastructure.
- [Amazon S3 API reference](https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html) — the de facto object storage API.
- [Amazon S3 User Guide: versioning, lifecycle, presigned URLs, Block Public Access](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html) — one provider's documentation of these features.
- [RFC 8881](https://www.rfc-editor.org/rfc/rfc8881) — Network File System (NFS) Version 4 Minor Version 1 Protocol.
- [RFC 8446](https://www.rfc-editor.org/rfc/rfc8446) — The Transport Layer Security (TLS) Protocol Version 1.3.
- [RFC 9110](https://www.rfc-editor.org/rfc/rfc9110) — HTTP Semantics.
- [NIST SP 800-57 Part 1](https://csrc.nist.gov/pubs/sp/800/57/pt1/r5/final) — Recommendation for Key Management.
- [SNIA Dictionary](https://www.snia.org/education/dictionary) — storage terminology.