Smart Scheduler (smart-scheduler.js)

The Smart Scheduler is the brain of the radio. It decides what plays next, builds the M3U playlists that Liquidsoap consumes, and does so with a blend of classical audio-engineering rules and machine learning.

In classical radio terms, this single module plays the role of the music director (what goes in the library), the programmer (the daypart and mood mix), the scheduler (rotation and clock building), and the continuity producer (imaging/ads/news placement) — all automated, all in one process, all sharing the same state. The sections below follow the same order a radio programmer reasons about their station: library → loudness → sequencing rules → adaptation → safety.

Role in the system

• Reads the media registry and category manifests.
• Produces playlists under PLAYLIST_ROOT (playlists/).
• Optionally fetches music (fetchFromYouTube) when a category runs thin.
• Applies loudness normalization and fingerprinting where needed.
• Keeps track of the current song / energy for live UI updates.

The scheduler is a pure decision maker: it never touches the audio stream itself (that is Liquidsoap's job), and it never decides whether to run (that is server.js, which owns the HTTP surface, the auto-refresh timer, and the pool-health monitor). This separation of concerns is what makes the system auditable — you can always ask "who last decided this" and the answer is one of: the scheduler (rotation build), the pipeline (library), or the server (refresh/supervision).

Toolchain

The scheduler resolves external tools from environment variables with Windows defaults:

Tool	Default path	Env override	Used for
FFMPEG	C:/ffmpeg/bin/ffmpeg.exe	FFMPEG_PATH	Transcoding, loudnorm, probe
FPCalc	E:/fpcalc.exe	FPCALC_PATH	Audio fingerprinting (duplicate detection)
yt-dlp	yt-dlp (on PATH)	YTDLP_PATH	Downloading music

Loudness normalization (EBU R128)

All assets are normalized to broadcast loudness so that songs, sweepers, jingles, and ads do not fight each other:

Target	Value
Integrated loudness	−14 LUFS
True peak	−1.5 dBTP
Loudness range (LRA)	11 LU

The normalization is performed with ffmpeg loudnorm filters. Two-pass: pass 1 measures the K-weighted integrated loudness $L$ (BS.1770 gated), pass 2 applies a constant linear gain,

$G = L_{target} - L_{measured}, \qquad L = -0.691 + 10 \log_{10}\!\left(\frac{1}{T}\int_0^T y_K^2(t)\, dt\right)$

then verifies the true peak on a 192 kHz oversampled signal, $TP = 20 \log_{10}(\max_n |x_{\uparrow 4x}[n]|)$.

Directories

On startup the scheduler auto-creates its working directories if missing:

• downloads/ — fresh downloads before they are moved into the library.
• archive/ — a per-category intake area for newly fetched music.
• weights/ — persisted reinforcement-learning weights.
• logs/ — scheduler run logs.

Library discovery

getFiles(root) is a recursive directory walker that returns all audio files below a root directory. The scheduler uses it to enumerate songs in every music category directory before scheduling, so new files dropped into a category folder are automatically eligible.

Critical safety fix

A music category with fewer than 15 tracks is automatically supplemented from the arabic_hits deck.

This guards against a stall during YouTube fetch windows: if a niche category is running dry, the scheduler keeps the station on air using the always-populated arabic_hits pool rather than hitting a silent gap.

Sequencing rules

The scheduler applies several constraints to each generated playlist. Think of these as the production clock of a traditional station — the rules a live programmer internalizes — encoded as hard constraints that never fatigue and never forget:

• Dirichlet sampling — the category mix is sampled from a Dirichlet distribution, so proportions wobble naturally over time but never collapse to a single category. With concentrations $\alpha_c = 6\,w_c$ derived from GENRE_SPLIT, each category probability is drawn as

  $X_c \sim \text{Gamma}(\alpha_c), \qquad p_c = \frac{X_c}{\sum_j X_j}$

  This keeps the hour varied while remaining on-brand. The "wobble" is the point: the long-run mix equals the configured split, but every individual hour is a different sample from that distribution — constraint-compliant variety rather than a fixed template (Blei, Ng, & Jordan, 2003). See Equations §2.

• Cooldowns — a track that has played will not be re-selected within its cooldown window:

  $\text{eligible} \iff (t - t_{last}) \ge \text{limit}, \qquad \text{limit} = \begin{cases} 90\ \text{min} & \text{songs (COOLDOWN\_PERIOD)} \\ 3\ \text{h} & \text{rotation refreshes (ROTATION.MUSIC\_FRESH\_MS)} \\ 10\ \text{min} & \text{sfx/jingles/sweepers/ads/news (ELEMENT\_COOLDOWN)} \end{cases}$

  This is the core anti-repeat mechanism. It is the software realization of the broadcast "no repeats within the hour" rule (Norberg, 1996; Keith, 2010) — generalized from a fixed clock position to a time-based cooldown, and hardened by persisted state and song-key dedup (so the same song downloaded as two files, or a song plus its "(TINI Version)" remix, cannot both air within the window). See Rotation Science.

• Energy sequencing — each track carries an energy rating (1–5). The scheduler avoids jarring jumps, e.g. it will not follow a 5-energy anthem with a 1-energy ballad unless the log genuinely calls for it. Energy is also the axis the mood arms and crowd-bias act on, so "pacing" is not a fixed pattern but a learned, live response to the audience.

• Artist adjacency — consecutive songs by the same artist are prevented so the same voice does not dominate back-to-back. This is ARTIST_GAP = 12 slots — a ~40-minute ceiling on how close two songs by the same artist can land, which is stricter than the "not back-to-back" rule of most traditional schedulers and is what kills the "Assala, Assala, Assala" clustering that a plain cooldown allows (the same artist's songs are different files, so a path-only cooldown can't see them as one).

• Category balance — long plays are broken by sweeper/jingle inserts to keep pacing radio-like. The cadence slots (AD_EVERY, NEWS_EVERY, MASHUP_EVERY, signature-jingle) guarantee the anchor elements air on schedule even though the music between them is stochastic.

Reinforcement-learning mood arms

The scheduler exposes three mood arms:

Arm	Mood
CHILL	Low energy, relaxed mix
BALANCED	Default energy distribution
PARTY	Higher energy, club-style pacing

Each arm is a Thompson-sampling bandit: the scheduler maintains a weight per category per arm, samples a score, plays the top result, then updates the arm's weight based on the realized reward (engagement / play quality). The chosen arm re-weights the category distribution for the next pass, so the station learns which mixes keep listeners. Thompson sampling chooses an arm with probability equal to its posterior probability of being optimal — the canonical explore/exploit balance (Thompson, 1933; Russo et al., 2018). See Rotation Science and Equations §9.

Neural scheduling (synaptic LSTM)

The scheduler ships an experimental neural path built with synaptic — a small LSTM-style network that can learn transition patterns from as-run history. It is an experimental variant layered on top of the rule-based engine; see Legacy & experimental variants.

Output

The scheduler writes standard M3U playlists (one per rotation) into PLAYLIST_ROOT. The production Liquidsoap worker reads the newest playlist and rotates files as it plays, while the scheduler keeps the next one ready. See Broadcast chain.

Anti-repeat & M3U rotation

The on-air rotation (the M3U file Liquidsoap watches) is deliberately stable. Liquidsoap uses reload_mode="watch", so every in-place write makes it reload the file and restart from the head — rewriting the M3U on every batch is exactly what historically caused "the same songs keep repeating". The rotation is therefore rebuilt only when it is actually consumed:

• rotationNeedsRefresh() — returns true when the buffer is empty, when rotationConsumed / buffer.length >= REFRESH_FRACTION (0.85), or when the daypart profile has changed since the rotation was stamped (see Daypart-stamped rotation below). Any other updateM3U() call short-circuits with Rotation stable — skipping rewrite and returns false without touching the file.
• markOnAirTrack() — called once per real on-air track (from the /metadata webhook in server.js). It advances rotationConsumed and stamps the just-played track in trackHistory so a future rotation never schedules it inside the fresh window.
• upcomingPaths(count) — returns the tracks still ahead in the rotation so content refreshes can protect them from purge (see below).

The 85% threshold deserves a moment's thought: it is not a magic number but the safety margin between "rotation is nearly done" and "rotation is empty." A rebuild + push + verify + Liquidsoap reload takes measurable time, and the rotation must never run dry while that happens. At 120 tracks, 15% headroom is ~18 tracks ≈ over an hour of airtime, which is far more than the rebuild ever needs. The daypart-OR makes the trigger faster when correctness demands it (never air the wrong daypart for hours waiting on consumption) while keeping the stability guarantee (never rewrite just because some external tick asked for it).

Double-reload head-replay fix (2026-08-10)

cat tmp > final on the m3u = truncate + write = two IN_MODIFY inotify events per push — Liquidsoap reloaded twice, restarting from the head each time and replaying tracks just aired. The fix:

1. smart-scheduler.js _writeM3uFile pads the m3u with blank lines to a fixed 32768 bytes (FIXED_M3U_SIZE; Liquidsoap's parser skips empty lines).
2. radio-engine.js remoteWrite writes .m3u files with dd if='<tmp>' of='<final>' conv=notrunc bs=1M — one write syscall = one IN_MODIFY, inode preserved. Every m3u is the same byte length, so dd-notrunc fully overwrites with no stale tail.

Verified live: remote main.m3u is exactly 32768 bytes (tail = \n padding), sort | uniq -d = 0 dups, and the Liquidsoap log shows exactly one Reloading playlist per push. Failed experiments (reverted — do not repeat): reload_mode="poll" and "off" reload ~1/s (catastrophic), and mv-rename breaks liq's inode watch (frozen rotation).

Anti-repeat state is persisted to E:\radionew\models\weights\scheduler_antirepeat.json (trackHistory pruned to 72 h, lastTrackPlayed, a 40-entry _recentOnAir, _lastMusicArtist, rotationConsumed) — saved on every markOnAirTrack() and rotation write, so a server restart never wipes cooldowns (the root cause of restart-replay). Beyond path cooldowns it also enforces:

• Song-key dedup — _songKey() normalizes artist|title (strips (...), [...], "official video/lyrics/remix/version" suffixes); push() rejects any track whose key already aired or is on cooldown via key, catching same-song-different-file and same-song-version pairs.
• Artist spacing — ARTIST_GAP = 12 slots: popFrom/push track artistPos, so no artist repeats within ~40 min of airtime.

Daypart-stamped rotation (2026-08-15)

The rotation is daypart-stamped so it can never drift across a profile boundary. _daypartKey() returns a canonical key for the active profile — deepnight (02–05) / morning (05–11) / midday (11–16) / evening (16–23) / latenight (23–02), or custom:start-end:hard:weights when SCHEDULER_CONFIG.DAYPARTS is set — mirroring _pickMusicCategory exactly so the key changes precisely when the build recipe changes:

$k(t) = \begin{cases} \text{deepnight} & 02 \le h < 05 \\ \text{morning} & 05 \le h < 11 \\ \text{midday} & 11 \le h < 16 \\ \text{evening} & 16 \le h < 23 \\ \text{latenight} & \text{otherwise} \end{cases} \quad\Longrightarrow\quad \text{needsRefresh} = \left(k(t) \ne k_{stamped}\right) \lor \frac{\text{consumed}}{\text{length}} \ge 0.85$

1. updateM3U() stamps this._rotationDaypart = this._daypartKey() on every successful build; the constructor initialises it to the current daypart (a fresh boot does not false-trigger).
2. rotationNeedsRefresh() returns true the moment the stamped key ≠ the live key (rebuild at the boundary, no need to wait for 85% consumption).
3. A 60s watchdog in server.js ([M3U] Daypart watchdog fired) runs a cheap daypartChanged() string compare and calls rotateBatch() within 60s of any boundary crossing.

This killed the deep-night "Arabic leaks": previously a 120-track mixed rotation built before 02:00 kept airing Arabic through the 02–05 English-only gate until it hit 85% consumed. See Equations §3.1.

Signature-jingle recurring cadence (2026-08-15)

loklok1_mixdown.mp3 ("LOKLOK FM") is the station signature and must recur every ~30–45 min, not appear once per rotation. The next airing slot is re-rolled uniformly after each firing,

$s_{k+1} = s_k + U, \qquad U \sim \text{Uniform}\{2,3,4,5\}, \qquad \mathbb{E}[U] = 3.5\ \text{slots}$

so the cadence is recurring but not metronomic:

1. SIG_JINGLE_NAME = 'loklok1_mixdown.mp3'; SIG_JINGLE_MIN_SLOTS = 2, SIG_JINGLE_MAX_SLOTS = 5 (override with SCHEDULER_CONFIG.ROTATION.SIG_JINGLE_EVERY_SLOTS for a fixed cadence).
2. The updateM3U jingles snapshot filters out the sig, so it can never air from the random pool — only via its reserved slots.
3. A reserved slot fires when imagingSlotsFilled >= nextSigAt; after each firing nextSigAt is re-rolled uniformly into [min, max] (mean ~3.5 slots ≈ every 30–45 min) — recurring but not metronomic. The repeatable push bypasses the path seen dedup so the same file can recur. Log: sigEvery=rand sigRange=2-5 sigNext=<target>.

Verified live: 7 loklok1_mixdown airings per 120-track rotation at varying spacing. See Equations §15.

Building a fresh rotation

updateM3U(force = false) regenerates the rotation from shuffled snapshots of the decks — it does not use drawCard(), because drawCard serves the Rulebook/supervision lookahead and only records into a separate, non-persisted _recentlyDrawn map (TTL-pruned); it never touches trackHistory or cooldowns. markOnAirTrack() (fed by the /metadata webhook) is the sole writer of trackHistory, so supervision draws can't burn airtime cooldowns. Building rules:

1. Least-recently-aired pop — popFrom scans the whole deck and picks the least-recently-aired eligible candidate: never-aired (last = 0) wins, then the longest-waiting track (tie-break = oldest addedAtOf). This fixes the "same songs after cooldown" recycle where the first off-cooldown track in fresh-first order kept re-airing the newest few. Tracks on cooldown (3 h music / 10 min imaging) stay in the snapshot and are skipped, so the 120-track rotation is drawn from songs the listeners did not just hear.
2. Strict pass — weighted category selection with a guard against a single starving category (e.g. the ~35% mashup weight on a small pool) blocking the rotation.
3. Refill pass — if the strict pass stalls, remaining slots are filled fresh-first from any off-cooldown music, then imaging.
4. Relaxed safety valve — only if fresh-first still cannot reach the target (tiny library / right after a purge), it falls back to on-cooldown tracks so Liquidsoap always gets a full rundown — but it never re-adds the just-played track (allowExclude is always false) and never duplicates.
5. Empty buffer is never written — an empty overwrite would make Liquidsoap emit Fetch failed: empty and stall; the previous file is kept instead.
6. Written via tmp + atomic rename so a failed write can never truncate the live file, then rotationConsumed/rotationWrittenAt are reset. The M3U is padded to a fixed 32768 bytes (FIXED_M3U_SIZE) so the remote dd-notrunc push is a single IN_MODIFY (see Liquidsoap).
7. Guaranteed imaging cadence — ads every AD_EVERY slots (~8th), news every NEWS_EVERY slots (~20–24th ≈ hourly), and mashups every MASHUP_EVERY slots (~8th, skipped in deep night), with a stinger-led bridge so mashups actually air instead of being converted to pure sfx.
8. Short-item crossfade override — every sfx/shorts/jingles/sweepers/ads/news entry is emitted with annotate:type=<type>,liq_cross_duration=<d>:<path> (CROSS_OVERRIDE: sfx/shorts 0.2s, the rest 0.5s) so Liquidsoap never tries to pre-buffer 2s of a 0.24s stinger. Music/mashups get no override. See Equations §12.1.

Top-of-hour NEWS deck (2026-08-11)

bulletin-engine.js generates a R128-mastered Arabic spoken bulletin (edge-tts ar-EG-ShakirNeural + Open-Meteo Cairo weather + news.txt headlines) into media/news/NEWS_<ts>.mp3 (~34–37s, Break-tagged). The scheduler treats it as imaging:

• news is a member of decks, categories, imagingCategories, isImaging, and the 10-min ELEMENT_COOLDOWN list, and is exempt from the 45s music quality-gate probe (a 35s bulletin would fail the music floor).
• NEWS_EVERY cadence slot (~every 20–24 tracks ≈ hourly) forces a news pick — same pattern as the AD_EVERY slot — so the bulletin airs top-of-hour in every daypart (deep night included).
• server.js auto-generates a bulletin at boot +30s and every BULLETIN_HOURS (default 1h) via scheduleBulletinAutoGen() (chained setTimeout, 0 disables). See Bulletin engine.

Purge protection

server.js liveProtectedPaths() protects nowPlaying + the lookahead queue + lastTrackPlayed + the next 40 rotation entries (upcomingPaths) from content refresh purges. Previously a purge archived mid-rotation files, Liquidsoap hit Fetch failed, and skipped to the surviving tracks — which repeated. After a refresh, resyncSchedulerAfterRefresh() forces updateM3U(true) so the rotation is regenerated from the post-purge library instead of pointing at archived files.

Social deck overlay

The scheduler supports a ranked overlay from the social platform:

• setSocialDeck(cat, posts) / clearSocialDeck() maintain this.socialDeck (empty {} by default).
• updateM3U prepends each category's ranked overlay to that category's snapshot before rotation building — an empty overlay is a zero-change path, so the scheduler's behaviour is identical when no social audio is approved.

socialPlatform.attachScheduler(sched) rebuilds the overlay at boot and inside resyncSchedulerAfterRefresh. See Social Platform.

Related

• Scheduler parameters — the tunable constants.
• Playback engine — how playlists are realized on air.
• Content manager — category manifests & catalogs.
• Gemini AI — AI-assisted content decisions.