· Protocolzone
A risk analyst on the operations desk types a partial account number into one box and expects back the player it belongs to, every open bet they hold, the markets those bets touch, and anyone else betting from the same card or device. On a platform running many operator brands at once, that lookup has to reach across bets, wallets, players, markets and tenants, over a window of weeks, and return while the analyst is still looking at the screen.
The store that accepts the bets cannot answer that well. This post is about the search index we run beside it to make queries like that fast, and about the one discipline that keeps the arrangement honest. We run this in production on a multi-tenant tote and fixed-odds platform we build and operate, so what follows is delivered work rather than a design sketch.
Why the system of record can’t answer these
The transactional store is tuned for one job: accept a bet and record a wallet movement, correctly, under load, keyed by identity. A wide-column store such as ScyllaDB or Cassandra is excellent at “give me bet 4f9c” or “give me the bets for player P”, because those read straight off the partition key. It is poor at “find every bet on this runner, placed in the last fortnight, from an account opened this month, across all tenants”. That is an arbitrary predicate over attributes that are not the key.
You can chase each of those queries with another secondary index, but every index you add is more work on the write path that has to accept bets when a market is live. So you keep the transactional store lean and deliberate, and you put the ad-hoc query load somewhere built to carry it. That somewhere, for us, is Elasticsearch.
The index is a projection, never the source of truth
This is the rule that everything else hangs off, so it is worth stating flatly. The index is not the system of record, it is a projection of it. The ledger owns the truth about money and bets. Elasticsearch holds a derived, query-shaped copy.
Three consequences follow, and treating any of them as optional is how teams get burned:
- If the index and the ledger disagree, the index is wrong, by definition, and the fix is to rebuild it — never to reconcile the ledger towards it.
- No money-affecting write ever lands in the index first. Search is a read model downstream of the transactional write, not a second place bets are created.
- Anything a payout or a balance depends on is read from the ledger at the point of decision. Search tells an operator which bet to look at; the authoritative figure comes from the store that settled it.
Hold that line and the index is free to be disposable, denormalised and optimised purely for reading. Blur it and you have built a second source of truth that will eventually contradict the first at the worst possible moment.
Indexing the bet and transaction flow
Bets, wallet movements and market changes already flow through Kafka as the platform’s event backbone. Indexing rides that stream: consumers read the domain events and project each one into an Elasticsearch document shaped for the query, not for storage.
The shaping is the point. A bet document carries the player, the market, the tenant, the runner and the current status denormalised onto it, so the analyst’s lookup is answered by a single query with no joins. The transactional store keeps those things normalised and separate because that is right for writes; the projection collapses them because that is right for reads. Same facts, two shapes, each fit for its own load.
Out-of-order and duplicate events are a fact of any streamed projection, so each document carries a version derived from the source event, and a later version never loses to an earlier one that arrived after it.
Reindexing, and staying consistent
Because the index is disposable, you must be able to rebuild the whole thing from the system of record on demand. You will need to: mappings change, an analysis chain gets tuned, a projection bug ships, or you simply want a clean copy. The pattern is to build a fresh index alongside the live one and swap an alias over once it has caught up, so readers never see a half-built index.
Consistency between ledger and index is eventual, and the honest engineering is in bounding the lag rather than pretending it is zero. The delay between a wallet movement committing and the projection reflecting it is a number you monitor and alarm on. It also sets a hard rule for the tooling built on top: an operator never makes a settlement or payout decision off the projection. The projection points them at the record; the record gives the number.
Relevance versus exact filters
Operator and compliance tooling is mostly exact filters and ranges: this tenant, this date window, this bet status, this stake band. Those need to be deterministic and auditable, because a compliance query that returns different rows depending on a relevance score is not a compliance query. Filters stay filters.
Relevance earns its place in two narrower spots. One is the free-text global box, where someone types a fragment and wants the most likely matches first. The other is fuzzy identity matching — a misspelt surname, a transposed digit in an account number — where typo tolerance is the difference between finding a colluding account and missing it. Keep the two modes separate in the query layer so that nobody accidentally ranks an exposure report by textual relevance.
Global search across entities
The single box resolves across entity types. A fragment can match a player, a bet reference, a market or a tenant, and the results come back typed and grouped so the analyst sees what each hit is before they click it. Type-ahead runs off the same index.
This is also where the platform’s risk cohorts become findable rather than discovered after the fact. Fraud rings, VIPs and bonus hunters share patterns: shared funding instruments, correlated staking, device overlap. A search index that has those attributes denormalised onto player and bet documents turns “who else looks like this account” into a query an analyst can actually run, instead of a report they wait for.
Search analytics, and where search stops
Elasticsearch aggregations carry the light analytical load that sits next to search: facet counts on a result set, a quick breakdown of open bets by market, the distribution behind a filtered view. That is search-based analytics, and it is genuinely useful for operator consoles.
It has a boundary, and naming it is what keeps the architecture clean. Heavy time-series and OLAP aggregation over the full history, meaning long windows, high cardinality and many group-bys, belongs in a columnar store built for it. For that we use Apache Pinot rather than pushing the search cluster into a role it is not shaped for. Three stores, three jobs: the ledger settles, Pinot aggregates history, Elasticsearch finds and summarises.
And search settles nothing. On the wagering platform the index never computes a dividend or a payout; tote dividends are declared by the external host the bet was routed to, and the money is the ledger’s. Search finds the bet. It does not pay it.
Where this has run
This is delivered. The search index, the Kafka-fed projection, the reindex-and- alias rebuild and the cross-entity operator search all run in production today on the multi-tenant platform we operate.
None of it is specific to wagering. The shape transfers to any operator console sitting over high-volume records: a lean transactional store, a denormalised search projection kept honest as a read model, and a separate columnar store for heavy aggregation. It fits mining sensor and event logs, insurance claims, trading fills and retail transactions equally well. The domain changes; the discipline that the index is a projection does not.
- elasticsearch
- search
- kafka
- operator-tooling
- data-engineering