SkillPilotAI

Object / Blob Storage vs Block vs File Storage

A cheap, almost unlimited way to store large unstructured files such as images, videos and backups.

18.1 Why is it used?

  • Amazon/Flipkart have crores of product images. If you keep them as BLOBs in the database, the DB becomes slow and expensive.
  • Object storage scales almost without limit and is very cheap per GB, so there is no worry about a full disk.
  • Storage like S3 gives 11 nines (99.999999999%) durability because data is copied across multiple AZs.
  • Every file has an HTTP URL, so you can put it behind a CDN and serve users directly, with no load on the app server.
  • Block and file storage have their own roles: block disks for databases, file storage for shared folders.

18.2 How it works

  • Object storage: data is stored in a flat namespace by 'bucket + key' (for example products/123/main.jpg), along with metadata. Access is through an HTTP REST API (PUT/GET).
  • Block storage: a disk is split into fixed-size blocks and attached to one server (AWS EBS, Azure Managed Disk). Low latency, best for DBs and the OS.
  • File storage: a shared filesystem over NFS/SMB with a folder-and-file hierarchy; many servers can mount it at the same time (AWS EFS, Azure Files).
  • Storage classes / tiers: Hot (S3 Standard), Cool (S3 Standard-IA), Archive (Glacier). Rarely accessed data goes to a cheaper class, moved automatically by lifecycle rules.
  • Pre-signed URL: the backend hands out a time-limited signed URL, and the client uploads/downloads directly to S3, bypassing the app server.
  • Multipart upload: a large file (for example a 5 GB video) is uploaded in parallel parts; if one part fails, only that part is retried.
  • Versioning and replication: versions protect against accidental delete/overwrite, and cross-region replication supports DR.
  • Objects are immutable: you cannot edit the middle of a file, you must rewrite the whole object.
  • Security: keep buckets private by default, keep server-side encryption (SSE) on, and grant access through IAM policies / pre-signed URLs.
Image upload and serving with a pre-signed URL
  1. 1.Seller app: asks the backend for a pre-signed URL
  2. 2.Backend: returns a time-limited signed URL
  3. 3.Browser: PUTs directly to the S3 bucket
  4. 4.S3 event: a Lambda/worker creates thumbnails and WebP
  5. 5.CDN: serves the image to users, fetching from S3 on a miss

File bytes never pass through the app server, so the server's bandwidth is saved

18.3 Tools you can use

ToolTypeHow it is used
AWS S3Managed object storageDefault store for images, videos, logs and backups; CDN with CloudFront, archive to Glacier via lifecycle.
Azure Blob StorageManaged object storageAzure's equivalent of S3; Hot/Cool/Cold/Archive tiers, served with Azure CDN/Front Door.
Google Cloud Storage (GCS)Managed object storageGCP's object store; Standard/Nearline/Coldline/Archive classes, a data lake together with BigQuery.
MinIOOpen-source S3-compatible storageUse MinIO when you need an S3-like API in your own data center / on Kubernetes; the same SDK code works.
AWS EBSBlock storageA disk attached to EC2; for MySQL/Postgres data files and the OS boot volume.
AWS EFS / Azure FilesFile storage (NFS/SMB)Multiple servers mount one shared folder, for example the shared uploads folder of a legacy app.

18.4 Comparison: Object vs Block vs File Storage

PointObject (S3/Blob/GCS)Block (EBS/Managed Disk)File (EFS/Azure Files)
How data appearsBucket + key + metadata, flat namespaceRaw disk blocks, the OS builds a filesystem on topFolders and files, NFS/SMB share
AccessHTTP REST API (PUT/GET)A disk attached to one serverNetwork mount, many servers at once
LatencyMilliseconds (10-100 ms)Sub-millisecond, fastestLow ms, slightly slower than block
ScaleAlmost unlimited, petabytesFixed size limit per volumeLarge scale, auto grow (EFS)
EditingOverwrite the whole objectUpdate any block in placeEdit inside the file is possible
CostCheapest per GBExpensive, you pay for provisioned sizeMost expensive per GB
Best forImages, videos, backups, logs, data lakeDatabases, OS disk, high-IOPS appsShared folders, CMS, lift-and-shift apps

18.5 When to choose which

  • Object storage: user uploads, product images, video files, backups, logs - any large files that are write-once read-many.
  • Block storage: data files of a database (MySQL, Postgres, MongoDB) and anywhere low-latency random read/write is needed.
  • File storage: when many servers need the same shared folder and the app expects a POSIX filesystem.
  • MinIO: when you need an S3 API on-prem or in a hybrid setup, or a replacement for S3 in local testing.
  • Send cold data (old invoices, logs older than 1 year) to the Glacier/Archive tier and the cost drops a lot.

18.6 Real-world example

Real world: Flipkart seller product image upload (a hypothetical scenario)

  1. The seller taps 'Upload image' in the app, and the app asks the backend for a pre-signed S3 URL (valid for 5 minutes).
  2. The browser PUTs straight to the S3 bucket; the image does not pass through the app server, so the server's bandwidth is saved.
  3. On upload, S3 sends an event notification; a Lambda/worker creates thumbnail and WebP versions and puts them back in S3.
  4. The database stores only the image key (products/98765/main.webp); the actual bytes live in S3.
  5. A customer opens the product page and the image comes from a CDN URL; on a cache miss the CDN fetches from the S3 origin.
  6. On Big Billion Day crores of image requests arrive, but 95%+ are served from the CDN cache, so S3 load stays low.
  7. After 90 days, old order invoices move to Glacier through a lifecycle rule, cutting the storage bill a lot.

18.7 Interview tips

  • Common mistake: keeping images/videos in a BLOB column in the database - DB bloat, slow backups, expensive.
  • Common mistake: leaving a bucket public - many data leaks happened this way; use a private bucket + CDN or pre-signed URLs.
  • Common mistake: trying to run a database on object storage - a DB needs block storage.
  • Interview tip: 'How will you edit a file in S3?' - answer: you cannot, you upload a new version. Numbers worth quoting: 11 nines durability, object latency 10-100 ms vs block sub-ms.