Skip to main content

Storefront search (search)

Meilisearch-backed catalog search. Owns the indexer that mirrors Catalog events into per-Sales-Channel indexes, the typed query bridge, the typeahead-popup feed, the analytics ingest for committed search phrases, and the LLM-augmented-search opt-in.

The module composition root subscribes to Catalog events (product.created.v1, product.updated.v1, product.archived.v1, attribute.updated.v1) and to Settings events (settings.value_changed) without imports into either module's internals.

Public surface​

Verb + PathAudiencePurpose
GET /api/v1/search/suggest?q=…&limit=Nstorefront (no auth)Typeahead popup feed; returns up to limit ProductSummary rows ordered by Meilisearch relevance, searched and named in the request language: the Accept-Language tags heaviest q first, each matched to the channel's languages (exactly, then by base language), else the channel's default
POST /api/v1/search/recordstorefront (no auth)Fire-and-forget analytics ingest; persists one verbatim row in search_phrase_records per committed search
POST /api/v1/admin/search/llm/toggleadmin (search:write)Cross-setting-validating wrapper around search.llm.enabled; refuses to enable with 400 LLM_CONFIG_INCOMPLETE when the embedder credential does not supply a Base URL, API key and model for every targeted channel
POST /api/v1/admin/search/reindexadmin (search:write)The admin Reindex products action: rebuilds every sales-channel index now and answers a per-run summary; 503 SEARCH_BACKEND_UNAVAILABLE when Meilisearch is down or refuses the call

The full search results page reuses GET /api/v1/catalog/products?q=… — catalog owns that contract, and the storefront /search route just re-exports CatalogPage. There is no parallel results-page contract.

Settings​

Seven settings live under the search group, registered by packages/modules/search/src/manifest.ts:

CodeTypeDefaultPurpose
search.enabledbooleantrueThe module's activation control (platform-wide): switches the popup, the result feed, the analytics ingest, the admin LLM and reindex screens and the indexer off; nothing is dropped
search.popup.suggestion_countnumber8How many products the storefront popup shows; 0 suppresses the popup but committed search still navigates
search.popup.minimum_query_lengthnumber3Minimum characters before the storefront fires a suggest request and before the recorder persists a row
search.llm.enabledbooleanfalseEnables Meilisearch's hybrid lexical + semantic search on this channel
search.llm.embedder_credentialscredential_ref (llm)""A reusable LLM credential configuration (Credentials screen); its Base URL, API key and model are the embedder's OpenAI-compatible endpoint, key and model (e.g. text-embedding-3-small). Required to enable LLM-augmented search
search.reindex_interval_minutesnumber10How often the worker rebuilds every channel index; 0 disables the periodic sweep (manual reindex stays available)
search.index_task_timeout_secondsnumber120How long to wait for one Meilisearch indexing task before reporting it as still running; giving up does not cancel the task

The popup and LLM settings are sales-channel-scoped via the existing Settings model. The storefront popup reads its count + threshold per request via the universal getter (SettingsService.get); on any read error (cache miss, SettingNotRegistered) the suggest service falls back to the manifest defaults so a Settings hiccup never 500s the popup.

Toggling search.llm.enabled on for a channel attaches a Meilisearch embedder to that channel's index. We register the embedder under the well-known name default with source: 'openAi'; the url parameter makes it work against any OpenAI-compatible endpoint (OpenAI itself, Azure OpenAI, Ollama's OpenAI shim, …).

The toggle wrapper (POST /api/v1/admin/search/llm/toggle) refuses to flip enabled=true (400 LLM_CONFIG_INCOMPLETE) for any channel whose search.llm.embedder_credentials reference does not supply all three of Base URL, API key and model. The error envelope's details[] lists every gap as <channelCode>.search.llm.embedder_credentials with the missing part, so the admin UI can highlight it.

The reactor (in SearchEventSubscriber) is belt-and-braces: a power user PUTting search.llm.enabled=true directly through the generic Settings admin route (bypassing the wrapper) does not poison Meilisearch — the reactor warn-logs and leaves the index alone if the embedder credential is incomplete.

Disabling never refuses; the reactor calls index.resetEmbedders() so the channel falls back to plain lexical ranking.

Meilisearch v1.11 gates embedders behind the vectorStore experimental feature; v1.13+ treats embedders as stable. Both are supported transparently — the admin endpoint does not gate on the Meilisearch version.

Analytics ingest​

Every committed storefront search lands one row in search_phrase_records for the future Analytics module to aggregate:

ColumnNotes
phraseVerbatim — no typo correction or LLM expansion
phrase_normalizedlower(trim(phrase)), maintained at insert time; supports case-insensitive aggregation without rewriting the verbatim phrase
sales_channel_idFK → sales_channels.id (ON DELETE RESTRICT — dropping a channel must surface as a deliberate decision rather than silently delete history)
result_countNumber of products the search returned; 0 for dead-end phrases
recorded_attimestamptz default now()

Indexes:

  • idx_search_phrase_records_aggr on (sales_channel_id, phrase_normalized, recorded_at desc) — supports the analytics-module's "phrases by channel and frequency over a window" query.
  • idx_search_phrase_records_recent on (recorded_at desc) — for ops dashboards.

The recorder is fire-and-forget (SearchPhraseRecorder.record(...)): it returns once the row has been queued, not once persisted, and swallows every exception via warn-log. The route hands off the promise without awaiting (void recorder.record(...)) and returns 202 { ok: true } immediately. The storefront response is therefore never delayed or failed by analytics persistence.

Below-threshold phrases (shorter than the channel's minimum_query_length) are silently no-op'd server-side as a belt-and-braces against a UI bug flooding the table with stub phrases.

Indexer + event subscriber​

SearchIndexer writes one document per product into a per-channel index named products_<channel_code>. Document shape carries the catalog product surface (id, sku, type, status, slug, primaryAssetUrl, categoryIds, categorySlugs, attributes, createdAt, updatedAt) and, per language the channel offers, texts.<language>: the product's effective name and description at that channel and language ('' for a deliberately empty override), its searchable text attributes and the labels of its searchable select options in that language. attributes holds the non-text attribute values only. The indexer also drives Meilisearch's searchableAttributes (sku, texts, attributes) + filterableAttributes from the live product_attributes.is_searchable + is_filterable flags, and its sortableAttributes from SORTABLE_ATTRIBUTES (see Sorting).

Language-restricted match. A query searches only the shopper's language: texts.<L>.name, texts.<L>.description, texts.<L>.attributes and texts.<L>.options, plus sku and the searchable non-text attributes, where L is the request's language resolved against the sales channel's languages (checked against them before it is used). A Polish shopper's query never matches English text. A product that is off at the channel and language is not returned.

A document carries no price. Prices are the viewer's: the query path resolves them from the price books (price_lists) when it hydrates the hits, so the index can never decide what a buyer pays. A per-buyer price cannot be indexed in any case: one index per sales channel and one document per product means pricing the document would multiply the corpus by the customer base.

Prices in search results​

  • Hits and suggestions are priced for the viewer — their organization, customer group and sales channel — through one batched resolveListingPriceDetails call per page or popup, the same resolution the catalogue listing uses, so a search hit, a catalogue card and the basket show the same price. Suggestions are priced in the request's currency.
  • A variant product shows its lowest variant price as a "from" price, and a charged sale carries the buyer's own price as the struck-through price.
  • With the price-list setting pricing.show_list_price on, a hit carries listPrice beside a lower buyer price, exactly as a catalogue card does.
  • Sorting or filtering by price (?sort=price, ?sort=-price, minPrice, maxPrice) is never answered by Meilisearch: the catalogue routes such a listing to PostgreSQL, where price_lists' precomputed price surface orders it by the prices the cards show for that viewer.

Sorting​

Meilisearch refuses a sort on any field outside the index's sortableAttributes, so the fields the query path may sort on are declared once, in SORTABLE_ATTRIBUTES (search-indexer.ts): createdAt and name. That constant is both what the indexer applies to every channel index and the alphabet buildSort may emit — its return type is built from it, so a sort naming a field the indexer never declared does not compile.

The listing contract's four sorts map as follows:

?sort=Meilisearch
relevance (or absent)none — the engine's ranking rules
-createdAtcreatedAt:desc
namename:asc
-namename:desc

The name is sorted in the shopper's language: the sort runs on texts.<L>.name, declared sortable for every language the channel offers, so the order a buyer sees is the order of the names they read.

The name a result card shows does not come from the index. Every hit is re-read from Postgres before the page is answered, and the card takes the name the catalogue's listing card shows for the request's sales channel and language: the effective name there (Channel × language → Channel → Global).

Upgrading an index built before the sort settings were applied needs a reindex: the settings were never applied (sortableAttributes was [] on every index, and every sorted listing was silently answered by the Postgres fallback), and createdAt was never written into a document. A full reindex applies both. That happens automatically on the next periodic sweep in the worker role (search.reindex_interval_minutes, default 10); a deployment that runs no sweep needs the operator step — the admin Reindex products action or pnpm run cli search reindex in the root of the instance. An attribute.updated.v1 refresh reapplies the settings but writes no documents, so it restores name sorting and not -createdAt.

For attributes flagged isSearchable AND with a select-style valueType (select, enum, multiselect), the indexer resolves the option labels in each channel language and writes them to texts.<language>.options, so a customer searching for the rendered text they see (e.g. "Czerwony") hits products whose raw value is the option key (e.g. "red"). The labels are read once per indexing run. Toggling isSearchable off removes the option labels on the next refresh within the existing event-driven cadence.

Upgrading to the per-language document (feature 174)​

The document's name, description, localizedNames and searchableOptions fields, and the text values inside attributes, are replaced by texts. Run one full reindex after the upgrade: the periodic sweep in the worker role does it on its own (search.reindex_interval_minutes); otherwise use the admin Reindex products action or pnpm run cli search reindex. Until an index has been reindexed — its searchable attributes do not yet include texts — the plugin's boot check reindexes it, and in the meantime it is queried as before, without the language restriction, and a warning is logged. The index size grows with the number of languages a channel offers.

SearchEventSubscriber keeps the indexes in sync:

EventHandler
product.created.v1indexer.upsertProduct(productId)
product.updated.v1indexer.upsertProduct(productId)
product.archived.v1indexer.deleteProduct(productId)
attribute.updated.v1indexer.refreshAttributeSettings()
settings.value_changed (when settingCode === 'search.llm.enabled')indexer.attachEmbedderForChannel / detachEmbedderForChannel

All handlers swallow their own errors — a transient Meilisearch outage does not break the catalog write path. The reserved fallback in the read path (catalog/routes.public.ts) keeps storefront search functional with a stale index until the next offline search:reindex.

Availability per channel and language​

Since product data scopes (specs/163-product-data-scopes/) a product can be switched on or off per sales channel and per channel language, carry a publication window, and have a channel-specific audience. The indexer asks catalog (catalogProductAvailabilityPort.describeChannelAvailability) instead of reading the product status:

  • a channel member is indexed when it is on in at least one language of the channel; archived and deleted products are on in none;
  • each document carries liveLanguages (the channel languages it is on in), publishFromMs / publishUntilMs (the effective window in epoch milliseconds; 0 and Number.MAX_SAFE_INTEGER when open), and the channel-effective visibility and allowedOrganizationIds. All five are filterable attributes;
  • a query adds a filter for the request language (the Accept-Language tags heaviest q first, each matched to the channel's languages — exactly, then by base language — the first one the channel offers, else its default), the window at the moment of the query, and the caller's audience, so a page is cut from available hits only. The allow-list is a filter input and is never retrieved into a response;
  • hydration asks catalog again (filterAvailable) for every hit, so a stale document cannot show a product that is no longer available.

About a minute after the first start following the upgrade, the worker process brings every existing channel index's settings up to date without a reindex: the five availability fields become filterable, so queries keep being answered by the index instead of falling back to PostgreSQL. It reads the settings of each index once per start and changes them only where one is missing. A document indexed before this change still has none of the new fields and is served as before. After upgrading, run one reindex per sales channel (the search_reindex bulk operation or the reindex CLI below) so the window and per-language filters become exact.

When the index does not answer​

The fallback stays a fallback — a public catalogue must not 503 because search is unhappy — but it distinguishes two facts that used to be fused into one:

  • unreachable — the engine is down, unroutable or timing out. Transient. catalog logs it per request and re-runs the listing through Postgres; nothing else is expected of anybody.
  • refused — the engine answered, in milliseconds, that it will not run this query: invalid_search_sort, index_not_found, a rejected API key. Deterministic, will not pass on its own, and a defect in how this module configured the index. The reason string the caller logs names Meilisearch's own error code, and search reports it itself at error level — once per code, because an index setting is a deployment fact and a public listing would otherwise report it on every request.

That distinction is the whole reason the defect could survive on five live indexes: every sorted listing was falling back to Postgres, and the only trace of it was a line reading meilisearch unavailable about an engine that was up and healthy.

Storefront integration​

The page header (storefront/components/Header.tsx) renders a <form action="/search" method="GET"> with a progressively-enhanced <SearchAutocomplete> client component inside it:

  • 200 ms debounce; AbortController cancels stale fetches.
  • Below the channel's minimum_query_length, no request fires.
  • Keyboard model: ArrowUp/Down to move selection, Enter to navigate to the selected suggestion, Escape to close, click outside to dismiss.
  • 503 from /search/suggest surfaces a "search temporarily unavailable" item in the popup without breaking the static form.
  • With JS disabled, the form posts ?q=… to /search natively (no JS required for crawlability).

The /search page (storefront/app/(catalog)/search/page.tsx) re-exports CatalogPage — the listing, filters, sort, pagination, and empty state are identical to /catalog. The only search-specific behaviour is the analytics fire-and-forget: when ?q= is present, the page awaits listProducts (so resultCount is meaningful), then void recordPhrase(...) BEFORE delegating to CatalogPage.

Reindex CLI​

search reindex walks every Sales Channel and pushes its public product surface into Meilisearch. Idempotent — safe to run after a fresh endora demo seed or whenever the index drifts from Postgres.

It is a command this module declares in its manifest.ts and the host runs, so it reindexes through the one SearchIndexer the composition holds rather than building a second one:

# In an instance, from its root:
pnpm run cli search reindex

# In a checkout of the Endora Commerce repository:
pnpm --filter backend run search:reindex
# or, addressing the host binary directly:
pnpm --filter backend exec tsx src/cli.ts search reindex

The body is packages/modules/search/src/backend/cli/reindex.ts.

Testing​

  • backend/test/contract/search/public-suggest.contract.test.ts (7 cases) — happy path, limit override, threshold, oversize, missing q, limit OOB.
  • backend/test/contract/search/public-record.contract.test.ts (7 cases) — happy path, default result_count, channel pinning, empty/oversize/negative validation, below-threshold no-op.
  • backend/test/contract/search/admin-llm-toggle.contract.test.ts (5 cases) — incomplete-config refusal (every embedder.* permutation), full-config success, disable always succeeds, unauthenticated → 401.
  • backend/test/integration/search/manifest-reconcile.test.ts (2 cases) — group + 6 settings seeded with correct defaults; idempotent re-apply.
  • backend/test/integration/search/embedder-reactor.test.ts (3 cases) — attach on enable=true, detach on flip-back-to-false, no event-fire when wrapper refuses the toggle.
  • backend/test/integration/search/event-subscriber.test.ts (existing) — catalog-event-driven indexing.
  • backend/test/integration/search/catalog-via-meilisearch.test.ts (existing) — env-flag-dispatched read path.
  • backend/test/integration/search/sort-order.test.ts (6 cases) — each of the four sorts served by Meilisearch (x-search-backend is the load-bearing assertion: Postgres answers all four, so the order alone cannot tell a served page from a fallback), the declared sortableAttributes, and the absence of the indexed price.
  • src/backend/services/search-sort-attributes.test.ts (4 cases, in this module's package) — buildSort emits only fields SORTABLE_ATTRIBUTES declares.
  • src/backend/services/search-degrade-observability.test.ts (3 cases, in this module's package) — a refused query names the engine's error code and is reported once; an unreachable one is not reported twice.

The last two need no services; the rest run against real Postgres + real Meilisearch.

Extension points​

  • Custom rankers — Meilisearch supports custom ranking rules per index; the indexer can push the rule set when the admin UI grows a per-channel weighting affordance.
  • Embedder-source enum — search.llm.embedder_source with values openAi | huggingFace | rest | userProvided would let a single channel pick a non-OpenAI-compatible provider without renaming the three credential settings.
  • Reserved fallback — already implemented: when Meilisearch is unavailable, the catalog read path degrades to Postgres ILIKE search via catalog-query.service.ts so the storefront is never fully broken.