Indexes Are Product Promises

An index looks like a database optimization. In production, it is more than that. It is a promise that some question deserves to be answered quickly enough for a user, workflow, operator, or product surface to depend on it.

The index is the visible technical artifact. The promise is the thing that needs review.

The thesis

Every index encodes a product promise: this read path is important enough to make writes, storage, operations, and deletion more expensive.

That does not make indexes bad. It makes them architectural. A system without the right indexes turns users into background jobs. A system with too many indexes turns every write into repayment for past product uncertainty.

The hard part is not knowing that indexes speed reads. The hard part is deciding which reads deserve to be fast.

The production pattern

A page becomes slow. A support tool needs another filter. A workflow needs to find the latest record by status. A dashboard wants grouping by date, owner, and lifecycle state. A batch job scans too much. An API endpoint grows from occasional use into a hot path.

Someone adds an index. Latency drops. The local problem is solved.

Then the write path slows down. Storage grows. Backfills take longer. Bulk updates become riskier. Deletes need more work. Replication lag appears during write bursts. A migration that used to touch one structure now maintains several. A query planner changes behavior after data distribution shifts.

None of this means the index was wrong. It means the index made a promise and sent the bill somewhere else.

The bill is easy to miss because read pain is visible to users and write tax is often distributed across the system. The page owner sees the improvement. The ingestion owner, migration owner, storage owner, and operator inherit the cost.

The trap

The trap is treating indexes as local fixes.

When a slow query appears, the obvious answer is to add the missing index. Sometimes that is exactly correct. But the deeper question is whether the product has made a durable promise that this access path matters.

Premature indexing can hide poor product prioritization. If every internal screen, optional filter, export, and speculative workflow gets a first-class index, the database becomes a catalog of unranked desires. The product has not chosen what matters. The storage layer has been asked to make everything feel important.

The opposite trap is austerity. Teams resist indexes because they fear write cost, then force common user actions through scans, caches, or manual waiting. That is not discipline. It is pushing product debt into latency.

Indexes need product judgment, not reflex.

The model

I review indexes through five taxes and one promise.

Read path is the promise. Which user action, API, report, operator flow, or background workflow becomes dependable because this index exists? How often is it used? What latency budget does it protect? Is the query shape stable, or is it one filter in a sea of changing requests?

Write tax is the extra work on every insert, update, or state transition that touches indexed fields. An index can make a rare read faster by taxing every write. That trade can be right, but it should be named.

Storage tax is the extra structure the system must store, compact, replicate, back up, and restore. A B-tree or similar ordered structure is not just metadata. It is data with lifecycle, locality, and maintenance cost.

Operational tax is the effect on migrations, backfills, query planning, hot keys, lock behavior, replication, and incident response. An index can change how the database behaves under load, not just how a single query behaves when tested in isolation.

Deletion tax is the cost of removing or anonymizing data everywhere it has been made easy to find. Systems that index by many dimensions can make retention and erasure harder. The right to find data quickly often creates the obligation to delete it carefully.

The review question is: is this promise worth these taxes over the lifetime of the feature?

Where this model breaks

This model can sound too deliberate for obvious cases. If a high-traffic endpoint is scanning a large table because a basic lookup lacks an index, the right move is usually to add the index and move on. Not every database fix needs a philosophy.

It also breaks in systems where the storage engine, workload, or data size makes index cost negligible for now. Small tables, short-lived data, analytical stores, and offline workflows have different economics. A strict product-promise review can become ceremony if the blast radius is tiny.

The counterpoint is that economics change. Small tables become large. Admin filters become operational dependencies. Temporary dashboards become executive rituals. The review does not need to be heavy, but the promise should be written down when the index supports a durable behavior.

What I do now

When someone proposes an index, I ask for a short index note. It usually fits in a few lines.

Promise: what action must become fast or predictable?
Query shape: which columns, ordering, filters, and cardinality matter?
Write impact: which write paths pay for it?
Lifecycle: how does this affect backfill, retention, deletion, and restore?
Proof: what query plan, workload sample, or production signal justifies it?
Exit: when would we drop it?

I also look for product ambiguity. If an endpoint needs ten optional filters across low-selectivity fields, the problem may not be indexing. It may be that the product has not chosen its primary retrieval model. Search, reporting, operational triage, and transactional lookup are different experiences. They should not all be squeezed through the same table design.

For important indexes, I want the owning team to watch both sides of the trade. The read path should get faster or more predictable. The write path should not quietly degrade. Storage growth should be expected. Backfills should have a plan. Deletion should still be possible without a heroic cleanup later.

The principal-engineer lens is to connect database mechanics to product intent. An index is not only a way to avoid a scan. It is a statement about what the system exists to answer quickly.

Closing takeaway

Before adding an index, name the product promise it protects and the write, storage, operational, and deletion taxes it creates.