Probability & Bayesian Updating at LOKLOK

This page collects the probability functions that actually run in the LOKLOK engine and then develops the Bayesian view of the station: what is already Bayesian today (the Thompson-sampling RL mood arms), and where a Bayesian updating layer could be added to make the station's beliefs about its listeners explicit, honest about uncertainty, and updated continuously from telemetry.

The intent is deliberately "design-first": the RL arms are live code (smart-scheduler.js), while several of the ideas in §4–§5 are proposals for a future engine revision. Every formula that is production code is marked LIVE; every proposal is marked PROPOSED.

It is a companion to Equations & Math Reference (the full closed forms), Rotation Science (the radio context), and Smart Scheduler (the implementation).

---

1. The probability functions already in the engine

LOKLOK runs four families of probability machinery. Three are distribution draws used for stochastic-but-constrained scheduling; one is the online Bayesian learning loop.

1.1 Dirichlet genre sampling (LIVE)

The category of the next music slot is sampled from a Dirichlet distribution with concentrations proportional to the configured GENRE_SPLIT weights. The Dirichlet is the conjugate prior of the multinomial — it is the "distribution over distributions" that keeps the long-run mix on target while every individual hour follows a different path (Blei, Ng, & Jordan, 2003).

$w_c,\quad \alpha_c = 6\, w_c,\qquad X_c \sim \text{Gamma}(\alpha_c),\qquad p_c = \frac{X_c}{\sum_j X_j}$

The Gamma draw uses the Marsaglia–Tsang algorithm for integer/real shapes and the Box–Muller transform for the normal part (Marsaglia & Tsang, 2000; Box & Muller, 1958). See Equations §2.

1.2 Beta–Bernoulli Thompson sampling (LIVE)

The three mood arms (CHILL / BALANCED / PARTY) are a Beta–Bernoulli bandit. Each arm carries a Beta posterior over "probability that this arm is rewarded":

$\alpha_{k},\ \beta_{k},\qquad \theta_k \sim \text{Beta}(\alpha_k, \beta_k)$

A Bernoulli reward $r \in \{0,1\}$ is observed after each on-air track, and the active arm's posterior is updated by the conjugate rule:

$\text{Beta}(\alpha + r,\ \beta + 1 - r)$

In the concrete engine the counts are updated by signal type — a listener count that grew is a success ($\alpha{+}1$), a drop is a failure ($\beta{+}1$), a hype vote is a strong success ($\alpha{+}2$), a chill vote a strong failure ($\beta{+}2$), and a flat count between 5 s polls is neutral (no update, to keep a steady audience from hammering $\beta$). The posterior is then sampled for each arm (sampleBeta = the ratio of two Gamma draws), the argmax arm wins, and the winning arm's config patch is blended into SCHEDULER_CONFIG at rate 0.15 so manual/agent overrides still dominate instantly.

This is textbook Bayesian updating: the prior is $\text{Beta}(1,1)$ (a uniform prior), each observation updates the posterior, and the posterior is the policy via Thompson sampling (Thompson, 1933; Russo et al., 2018). Priors are persisted to E:\radionew\models\rl_arms.json so the belief state survives restarts — the engine never forgets what it has learned. See Equations §9.

1.3 Anti-fatigue proportional control (LIVE)

Not Bayesian, but a probability-family neighbor: the scheduler tracks observed category frequency $\hat p_c$ over the last 12 plays and applies a proportional correction to the target weight:

$w_c' = \max(0.02,\ w_c + (w_c - \hat p_c) \cdot 0.5)$

This is a classical negative-feedback controller (Åström & Murray, 2021), the same update family a Bayesian model would use as its observation model — §4 shows how the two can be unified. See Equations §4.

1.4 The NN/VAE scoring blend (LIVE)

Track selection scores candidates with a 0.7/0.3 blend of a trained LSTM success predictor and a station-vibe VAE (see Equations §7). These are discriminative models — they output point estimates, not posteriors. A Bayesian treatment of the same objects (Gaussian-process or MC-dropout uncertainty over the score) is one of the §5 proposals.

---

2. The Bayesian view of a radio station

A Bayesian formulation makes the station's decision process explicit as a belief-update loop:

The three terms every Bayesian system needs map cleanly onto LOKLOK:

Term	Meaning	LOKLOK equivalent
Prior	belief before data	Beta(1,1) uniform; config GENRE_SPLIT defaults
Likelihood	how data is generated given the state	Bernoulli "reward": listeners grew / hype vote
Posterior	belief after data	rl_arms.json α,β counts; the sampled θ per arm

What is still missing to call the whole station Bayesian (rather than just the arm selector) is a single joint model that treats listener count, category, daypart, and track features as one graph. §4–§5 develop that.

---

3. Why Bayesian updating is the right frame for a live station

Three properties of the live radio problem make the Bayesian frame a natural fit rather than a fashion:

1. Data is sequential and cheap. Every track produces a listener-count sample and (occasionally) a vote. The natural estimator for "does this policy retain listeners" is a recursive filter, not a batch re-fit — exactly the conjugate-update structure of §1.2.
2. Uncertainty is information. A bandit needs to know how sure it is, not just the point estimate. A policy played 3 times deserves more exploration than one played 3,000 times; the Beta posterior's variance encodes this automatically, and Thompson sampling turns that variance into action.
3. Non-stationarity is the default. Radio audiences shift by daypart, season, and trend. A static optimum does not exist; a posterior that is continually updated (and whose old evidence can be discounted) tracks a moving target. The engine's neutral-observation suppression (skip updates when the count is flat) is itself a variance-control choice that prevents noise from masquerading as signal.

The reader should note the difference between Bayesian updating (recursive posterior refinement, cheap, online — what LOKLOK's arms already do) and Bayesian modeling (a generative joint model with priors over everything — the §5 proposals). The former is live; the latter is a roadmap.

---

4. Proposal: a unified listener-count Bayesian layer (PROPOSED)

The RL arms already consume a binary "grew/dropped" signal. A richer, still cheap upgrade is to model the listener count itself as a Poisson-Gamma conjugate model per (arm, daypart) cell.

4.1 The Poisson–Gamma model

Listeners arrive over a polling window; the natural generative model is $\lambda_{k,d}$, the expected listener count for arm $k$ in daypart $d$:

$\lambda_{k,d} \sim \text{Gamma}(a_0, b_0), \qquad n_{k,d} \sim \text{Poisson}(\lambda_{k,d})$

Because Gamma is conjugate to Poisson, observing $n$ listeners in one poll updates a cell's posterior in closed form:

$\text{Gamma}\big(a_0 + n,\ b_0 + 1\big)$

The posterior mean $\mathbb{E}[\lambda] = a/b$ is the smoothed listener-count estimate — a recursively computed, daypart-aware average that never needs to store raw history. Comparing posteriors across arms within the same daypart then yields a per-daypart arm ranking with calibrated uncertainty, which is strictly more informative than the current single global arm switch.

4.2 Hierarchical daypart pooling

A hierarchical version borrows strength across dayparts (Gelman et al., 2013): each cell draws its rate from a global distribution,

$\lambda_{k,d} \mid \mu_k,\ \kappa \sim \text{Gamma}(\mu_k \kappa,\ \kappa)$

so a daypart with little data inherits the arm's overall rate, and the global $\mu_k$ is itself updated by every cell. In practice this means the 03:00 audience's tiny sample does not over-fit its own daypart cell — it is shrunk toward the station-wide arm behavior, which is exactly what a professional programmer does implicitly ("early mornings behave like overnights").

4.3 Decay: the non-stationary conjugate

Raw conjugacy assumes a stationary rate. Radio is not stationary, so the proposal adds an exponential discount of past evidence — the standard trick for conjugate models under drift. After each update the counts are shrunk toward the prior:

$a \leftarrow \rho\, a + (1-\rho)\, a_0, \qquad b \leftarrow \rho\, b + (1-\rho)\, b_0, \qquad \rho \in (0.9, 0.99)$

A decay of $\rho = 0.95$ gives a half-life of roughly 14 observation windows — recent audience behavior dominates, ancient history fades, and the posterior never collapses into a point mass (which would kill exploration). The existing observe() neutral-suppression stays: flat counts should still not add evidence.

---

5. Proposal: Bayesian track scoring (PROPOSED)

The current track score is a point estimate from an LSTM + VAE blend (Equations §7). A Bayesian scoring layer would replace the point with a posterior predictive over "success" for each candidate track:

$\hat y_t = \mathbb{E}_{p(\theta \mid \mathcal{D})}\big[\, f_\theta(\text{features}_t) \,\big]$

with a confidence band. Two practical routes, in increasing order of effort:

1. MC-dropout on the existing LSTM — run the network $M$ times with dropout active and treat the variance across passes as epistemic uncertainty. Near zero new code, a principled uncertainty per track.
2. Gaussian-process surrogate over a small feature vector (energy, category, time-of-day, recency) — a GP gives exact posterior predictive variance and a natural acquisition function (e.g. "play the track with the highest lower-confidence-bound-adjusted score"). GPs are the standard tool for Bayesian optimization of expensive objectives; the "objective" here is listener retention per candidate.

The scheduling benefit: when two tracks score near-identically, the system can prefer the one with lower uncertainty (exploit the reliable pick) or higher (build evidence) according to a tunable exploration schedule — a principled replacement for the fixed 0.7/0.3 blend and the flat anti-fatigue gain.

---

6. What is live today vs. what is proposed

Component	Status	Where
Dirichlet genre draw	LIVE	smart-scheduler.js _pickMusicCategory
Beta–Bernoulli Thompson arms	LIVE	ReinforcementLearningEngine, rl_arms.json
Anti-fatigue proportional control	LIVE	smart-scheduler.js category-weight correction
NN/VAE point-score blend	LIVE	TrackSuccessPredictor + VAE
Poisson–Gamma listener layer	PROPOSED	§4.1
Hierarchical daypart pooling	PROPOSED	§4.2
Exponential evidence decay	PROPOSED	§4.3
Bayesian (MC-dropout / GP) track scoring	PROPOSED	§5

The live core already performs genuine Bayesian updating in production, 24/7, with a persisted belief state. The proposals generalize the same machinery from "which mood arm" to "how many listeners, in which daypart, with how much confidence" — the same closed-form conjugacy, applied more broadly.

---

7. FAQ

Q: Is the station's learning already Bayesian? A: The mood-arm selector is: Beta–Bernoulli conjugacy with a uniform prior, posterior sampling, and persisted beliefs (rl_arms.json). The rest of the scheduler (genre draw, anti-fatigue, NN/VAE scoring) is stochastic + learned, but not Bayesian.

Q: Why Beta(1,1) as the prior? A: It is the uniform prior on [0,1] — no arm is preferred before data. With listener counts of tens, α and β grow quickly and the prior influence disappears within hours, so the choice is behaviorally neutral.

Q: Why neutral observations update nothing? A: Because a flat count between 5 s polls is evidence about the count, not about the arm. Updating β on every flat poll would punish the current arm for a steady audience. The existing observe() guards exactly this (see Equations §9).

Q: Would the Poisson–Gamma layer change scheduling? A: Not the draw mechanism — it changes the reward signal the arms consume (from a binary grew/dropped to a smoothed per-daypart rate) and would let the scheduler tune per-daypart weights from data instead of the fixed multipliers. Both are backward-compatible changes to observe().

Q: What about a fully Bayesian prior over GENRE_SPLIT? A: That is the natural end-state: a Dirichlet prior over the category weights per daypart, updated by realized category frequencies — turning §1.1's static Dirichlet draw into the posterior predictive of a learned Dirichlet-multinomial model. It is the cleanest of the proposals and a direct upgrade of live code. See Equations §2 for the current static form.

---

Related

• Equations & Math Reference — the closed-form mathematics (§2 Dirichlet, §9 Thompson arms).
• Rotation Science & Comparison — how the scheduler compares to a traditional pipeline (§3.3 covers the arms).
• Smart Scheduler — the implementation.
• References — Blei, Marsaglia–Tsang, Box–Muller, Thompson, Russo et al., Åström & Murray, Gelman et al.