# discovery-by-llm

Music discovery in a chat form: describe the moment, get verified Spotify
tracks with a reason per track, save the result as a playlist.

Spotify removed its recommendation and audio-intelligence endpoints for new
apps, so in this PoC the LLM fills that role. It interprets the request and
proposes candidates; Spotify verifies them and supplies the taste profile,
album art and playback. The eval suite compares the output against bare
Spotify search on the same questions.

> Chronological build log, including what was dropped and why (Dutch):
> [docs/logboek.md](docs/logboek.md).

## Running it

**Hosted instance** (credentials in the submission mail): a live deployment
bound to my own Spotify account, with real personalisation and playlist
writes. Spotify dev mode caps an app at 5 allowlisted users, so a reviewer
cannot log in with their own account; I can allowlist a reviewer account on
request.

**Without any keys:**

```
docker compose up --build
```

Demo mode replays recorded API cassettes through the same pipeline and
streaming path as live mode. The suggestion chips map to the recorded
scenarios; any other input replays the nearest scenario, with a banner
naming it. Playlist writes are simulated and labeled.

**Live with your own keys:** copy `.env.example` to `.env`, set
`APP_MODE=live`, a Spotify client id (redirect URI
`http://127.0.0.1:8888/callback`) and an Anthropic API key, then
`docker compose up --build` and log in via Spotify.

## How it works

```
query ──► call 1: intent + candidates ──► grounding ──► call 2: rerank ──► cards
            (LLM, structured output)      (Spotify        (LLM, streamed
                    ▲                       search          per track)
            taste profile                   fan-out)            │
            (cached, /me/top + saved)          │           playlist write
                                          verified pool
```

- **Call 1** interprets the request (mood, activity, era, language,
  familiarity) and proposes 35 candidate tracks, informed by a compressed
  taste profile fetched once per session.
- **Grounding** resolves every candidate against Spotify search in a
  bounded concurrent fan-out with early stop, a request deadline and a
  name-to-id cache. Unverified candidates are dropped. Misses and
  mismatches are logged as separate rates.
- **Call 2** reranks the verified pool and streams one justification per
  track as NDJSON events; cards render as they arrive.
- Only track ids from the verified pool can reach the user. In live
  testing at familiarity `new`, the model fabricated 12 of 32 candidates;
  the matcher dropped all 12 and the user still received 15 verified
  tracks.
- An honest limit on discovery depth: "new to you" means "not in your top
  or saved tracks", and the intent prompt favors well-known tracks from
  adjacent scenes because those verify reliably (chasing obscure work is
  where the model starts inventing titles). A listener deep in those
  scenes will recognize a fair share of the results. Going deeper needs a
  repair loop and a better novelty signal; see the last section.
- Refinement turns rerank the existing pool; turn 2 makes zero Spotify
  calls.

## Choices

- **Two LLM calls around a deterministic grounding step**, no agent loop:
  predictable latency and every stage testable on its own.
- **Hand-rolled Spotify client** (~200 lines over httpx). The endpoint
  surface is small and the retry, rate-limit and caching behavior is where
  the engineering lives. Retry policy is per endpoint: bounded Retry-After
  honor on reads, one token refresh, and no retries on playlist writes
  (side effects would be ambiguous).
- **Ports and adapters.** Demo mode is a second implementation of the same
  ports. The domain model never sees Spotify JSON.
- **NDJSON over a streamed POST.** The request has a body and EventSource
  cannot POST. Typed events (metadata / track / warning / error / done)
  with a strict client-side phase machine.
- **Degradation is explicit.** Below the grounding floor the app returns
  what resolved plus a warning; quota exhaustion (QUOTA_EXCEEDED) is
  recognized and never retried; live mode never falls back to fixtures.
- **Hosted seed session.** The public instance installs a session from an
  escrowed refresh token at startup, so every visitor shares my account,
  including playlist writes. HTTP basic auth at the edge is what makes
  that acceptable. Playlists carry a fixed name prefix for bulk cleanup.
- **No embeddings.** Search is the only entry point into the catalog; an
  index does not pay for itself inside a PoC.

## Performance

Time to first card is 20 to 30 seconds, measured through the UI and in the
eval suite; the spread is LLM latency. Streaming keeps the wait honest:
metadata arrives early, cards render one by one, progress states name what
is happening.

Under the hood:

- Search `limit` is capped at 10 since Feb 2026, so resolving 35 candidates
  requires a fan-out: bounded concurrency, early stop once the rerank pool
  is full, request deadline, partial results on quota exhaustion.
- Name-to-id resolution cache and a session-scoped taste-profile cache.
- Anthropic prompt caching between calls.
- Per-request counters (Spotify calls, cache hits, LLM tokens) in
  structured logs, surfaced in the dev panel.

Bare search matches words; it has no notion of mood, era, energy or who is
asking. The pipeline answers from the listener's actual scenes. Across all
8 eval scenarios the pipeline delivered 11 to 15 verified tracks from 8 to
20 distinct artists; on the discovery scenario, all 15 were outside the
listener's top and saved tracks on Spotify. A sample (full table via
`eval/run_eval.py --baseline`; the eval checks are plain code assertions,
no LLM judging):

| "Energy for the gym" | Pipeline                                 | Bare search                                                    |
| -------------------- | ---------------------------------------- | -------------------------------------------------------------- |
| 1                    | Baddadan by Chase & Status, Bou, Flowdan | Redbone (with GloRilla) by Lil Baby                            |
| 2                    | Tough Talk by Chase & Status, Kwengface  | Rock That Body by Black Eyed Peas                              |
| 3                    | Solar System by Sub Focus                | Instigator by Inpatient, Ren, Chris Webby                      |
| 8                    | DJ Turn It Up by Dimension               | Gym Power Beat by Ultimate Fitness Playlist Power Workout Trax |

## Where to look

| Criterion                  | Where                                                                                                  |
| -------------------------- | ------------------------------------------------------------------------------------------------------ |
| Code quality and structure | `backend/app/` (domain / ports / adapters / pipeline / api), `AGENTS.md`                               |
| AI application             | `backend/app/prompts.py`, `backend/app/adapters/anthropic/llm.py`, `backend/app/pipeline/grounding.py` |
| Spotify integration        | `backend/app/adapters/spotify/` (PKCE, retry policy, mapping)                                          |
| Performance                | `backend/app/pipeline/grounding.py` (fan-out, cache, early stop), `backend/app/observability/`         |
| Streaming contract         | `backend/app/api/recommendations.py`, `frontend/src/lib/recommendationStream.ts`                       |
| Evaluation                 | `eval/` (runner, scenarios, baseline arm, recorded fixtures)                                           |
| Pragmatic cuts over time   | `docs/logboek.md`                                                                                      |

## Method

About one hour of preparation (API research, a design spike) and about
eight hours of hands-on build time across three days. The build was
agent-assisted: coding agents implemented scoped tracks from briefs I
wrote, while I owned the architecture, the API contracts, every code
review, every merge and all live verification against the real APIs. Work
landed in reviewed batches, which is why commits arrive in bursts. The
logboek documents the process as it happened; the setup behind it (server,
Forgejo/GitHub mirror, deployment, how the agents are used) is mapped in
[docs/workflow.md](docs/workflow.md).

## What I cut, and what would come next

- A faster model on the intent call, which would cut most of the time to
  first card; the grounding stage already measures the trade-off to watch
  (fabrication rate), so the experiment is cheap to run safely.
- A guaranteed minimum grounding budget. The request deadline is anchored
  at request start, so an unusually slow intent call can leave grounding
  with no time at all; raising the deadline works around it (env-tunable),
  but grounding deserves a floor of its own regardless of what call 1
  spent.
- Deeper discovery. A repair loop that re-prompts the model for candidates
  that failed verification would let the prompt chase less obvious work
  without losing the pool, and a novelty signal beyond exact-id exclusion
  (artist-level familiarity, recently played) would push results past
  "famous in an adjacent scene".
- Refinement turns that re-ground; today turn 2 reranks the existing pool.
- Chat history persistence. Conversations are client-side only and the
  server keeps no conversation state; sessions and caches are in-process,
  Redis is the named production step.
- Playback queue control, an i18n framework (copy is already centralized),
  per-stage byte snapshots in the eval, and a semantic index over grounded
  candidates.
