Search Engine - Full-Text Search

Finding relevant results in milliseconds, based on the words the user typed, out of crores of products/documents.

SQL's LIKE '%red shoes%' does a full table scan on crores of rows, is very slow, and gives no relevance ranking.
Users misspell words (iphnoe) or use synonyms (mobile/phone); a search engine handles fuzzy matching.
Filters and facets are needed: brand, price range, rating, size - with fast counts through aggregation.
Results must be ranked by relevance (BM25 score), popularity and stock.
The same technology is used for log search (ELK stack).

Inverted index: for each word, a list of the documents it appears in, for example 'shoes' -> [doc1, doc7, doc99]; like the index at the back of a book.
Analyzer: tokenizes text, lowercases, removes stopwords and applies stemming (running -> run) before indexing; the same analyzer runs on the query too.
Relevance scoring: BM25 (an improved TF-IDF); documents with rare words and more matches rank higher.
Sharding and replicas: the index is split into shards spread over nodes; replicas give read scale and HA.
Near real-time: a new document becomes searchable after the refresh interval (default ~1 sec).
Sync from DB: the DB stays the source of truth, and changes flow to the search index through CDC (Debezium) or Kafka.
Fuzzy search, synonyms, facets/aggregations, highlighting - the main features of e-commerce search.
Vector/semantic search: meaning-based search through embeddings (kNN); hybrid (BM25 + vector) is common these days.

A. Indexing: CDC sync from DB to search index

The product is searchable after the ~1 sec refresh interval

B. Analyzer + Inverted Index (like a book's back-of-book index)

SQL LIKE '%red shoes%' = full table scan, no ranking

C. Query flow: 'nike runing shoes under 3000'

1.User query: 'nike runing shoes under 3000'
2.Query parse: fuzzy runing -> running, filter price <= 3000
3.ES coordinator: sends to every shard in parallel + brand facet agg
4.ES cluster: Shard 1, Shard 2, Shard 3 ... Shard 100, each returns local top hits (50 crore products, 100+ shards + replicas for read scale)
5.Merge + re-rank: BM25 + popularity + stock + sponsored
6.Response: top 20 results, ~100 ms

On sale day, add replicas and use a query cache

Tool	Type	How it is used
Elasticsearch	Open-source(ish) distributed search engine (Lucene)	Product search, log search (ELK); JSON REST API, sharding, aggregations; also available managed as Elastic Cloud.
OpenSearch	Open-source fork of Elasticsearch (Apache 2.0)	AWS's managed OpenSearch Service; forked from ES 7.10, almost the same API, for logs and product search.
Apache Solr	Open-source search server (Lucene)	Older and mature; enterprise/document search, distributed through SolrCloud.
Algolia	SaaS hosted search	Super-fast typo-tolerant search-as-you-type, ranking tuned from a dashboard; pay per search/record.
Azure AI Search / Vertex AI Search	Cloud managed search	Managed search services on Azure and GCP, with AI/semantic search.
Meilisearch / Typesense	Open-source lightweight search	Algolia-like ease for small-medium apps; can be self-hosted.

Point	Elasticsearch	OpenSearch	Solr	Algolia
Type	Lucene based, self-host or Elastic Cloud	ES fork, self-host or AWS managed	Lucene based, self-host	Fully managed SaaS
License	AGPL/SSPL/Elastic license options	Apache 2.0, fully open	Apache 2.0	Proprietary
Speed / latency	Fast, depends on tuning	Like ES	Fast	Very fast, ~ms, edge optimized
Scaling	Shards + replicas, you manage it	Same, managed on AWS	SolrCloud + ZooKeeper	Automatic
Analytics / logs	Very strong (Kibana)	Strong (OpenSearch Dashboards)	Okay	No, search only
Ops complexity	Medium-High	Medium (lower when managed)	High	Very low
Cost	Infra cost, cheap at large scale	Infra / AWS cost	Infra cost	Usage based, expensive at large scale
Best for	Product + log search, custom ranking	AWS shops, need an open license	Legacy/enterprise doc search	Startups, instant-search UX

Elasticsearch: a large e-commerce catalog, complex ranking and aggregations, and a team that can handle ops.
OpenSearch: you are on AWS and want a managed service + open-source license.
Solr: you are already on Solr or have an old enterprise document-search setup.
Algolia: you want to launch quickly, need the best search UX, and volume is moderate.
Small app that already has Postgres: to start with, Postgres full-text search (tsvector) may be enough.

Real world: Product search like Amazon/Flipkart (a hypothetical scenario)

A seller adds a new product, and a row is inserted in MySQL (the source of truth).
Debezium CDC sends the change to a Kafka topic, and an indexer service indexes the product into Elasticsearch.
After ~1 second the product becomes searchable; 50 crore products are spread over 100+ shards.
The user types 'nike runing shoes under 3000'; the analyzer tokenizes, and fuzzy matching turns 'runing' -> 'running'.
The query is: text match + filter price<=3000 + brand facet aggregation; each shard searches in parallel.
Results are re-ranked with BM25 score + popularity + stock + sponsored boost, and the top 20 return in ~100 ms.
On sale day, read replicas are increased, and popular queries are cached.

Common mistake: making Elasticsearch the primary database - it is a search index; keep the source of truth in the DB.
Common mistake: creating too many small shards (oversharding) hurts cluster memory and performance. Not planning the mapping is also wrong: if text vs keyword is wrong, filters/sorting do not work.
Interview tip: be sure to explain the inverted index and the DB -> CDC -> search index sync flow.
Worth quoting: refresh interval ~1 sec (near real-time), BM25 ranking, and the full scan of LIKE '%x%' vs an index lookup.