Skip to content

SIMUT — Concurrency Invariants

Companion to the stability plan. Every rule below is enforced (or made enforceable) by stability wave 1; PRs must not violate them. Referenced from code comments as docs/CONCURRENCY.md.

The model in one paragraph

Core 0 runs everything except the display, cooperatively, and owns the watchdog. Core 1 runs only DisplayManager::loopCore1( ) and is a multicore_lockout victim: LittleFS flash writes stall it via flash_safe_execute, which also disables IRQs on Core 0 for the duration of each program/erase — that window is what starves the CYW43 radio and is now measured by the FLASH_OP metrics (show metrics). For heavy saves, Core 0 puts the display in quiet mode: since wave 1, it first asks Core 1 to park at the top of its loop (a point outside malloc, the event-queue spinlock and any SPI burst) and only then hard-resets it; the park wait, CORE1_QUIESCE_MS (1.2 s), falls back to the old immediate reset, so behavior can never be worse than before. The alpha has no quiet mode, and its requestQuietMode( ) says so by returning false: a heavy save then pauses Core 1 around each flash write, the way the history and the log are written. Its loop parks at the top like the TFT’s. Until 2026-10-03 it did neither: the call answered true with Core 1 still running from the flash being written, and every other pause waited out the full park timeout.

Invariants

  1. Core 1 never touches flash/LittleFS. Language/license loads run on Core 0 only (DisplayManager.cpp, setLanguage).
  2. Core 1’s steady state is heap-free, log-free and queue-gated. No String/new in render/touch paths (wave 1 removed the per-frame offenders); the single LOG_CODE in DisplayManager runs on Core 0 (forceUnpause); every UI event goes through pushUiEvent( ), which drops events while quiesce is pending so the queue spinlock can never be held at reset time.
  3. Never hold _stateMutex across any FLASH_OP/LittleFS call. Core 1 blocks on that mutex; a flash lockout would then deadlock. Setters stay short; wave 2 adds a debug assert.
  4. Every request* has a matching release*. Quiet mode is refcounted; the T1.5 watchdog force-drains a leaked refcount after 15 s and logs APP_DISPLAY_PAUSE_STUCK param=1.
  5. Working static buffers in Web/Telemetry/Graph/Storage are Core-0-only and non-reentrant. They exist to spare the ~4 KB stack; never touch them from IRQs or Core 1.
  6. Core 1 is reset only after a confirmed park or an explicitly logged timeout (DSP_FORCE_UNPAUSE param=1 = quiesce timeout fallback).
  7. Long flash work is sliced. enforceStorageLimit( ) deletes at most 2 files per call; the remainder drains via update( ) one file per ≥15 s slice (STO_ENFORCE_BUDGET “deferred” marks the handoff).
  8. Anything reached from a flash_range_* call path lives in SRAM. src/FlashIrqProbe.cpp wraps the two SDK flash primitives via -Wl,--wrap, so its shims sit on the OTA applier’s path too — and ota_applier_run( ) erases the whole application slot while running from SRAM. A shim in the app slot would be fetched from an erased sector on the next call and brick the device mid-update. Both shims are __not_in_flash_func, they read timer_hw->timerawl directly instead of calling time_us_32( ), and they stay in 32-bit arithmetic so no flash-resident libgcc helper is reached. tools/check_flash_probe.py fails the build if either shim links outside 0x2000_0000–0x2004_2000.
  9. The watchdog window cannot be widened — bound the work instead. The RP2040 load register saturates at WATCHDOG_TIMEOUT_MS = 8388 ms, so every WdtWindow asking for 30 s or 120 s gets exactly the default. The class is still useful (a shorter nested window works, and the save/restore is correct) but it buys no time. Two corollaries, both learned the expensive way:
    • Feed synchronously, from the thread doing the work. An interrupt timer feeding on the side does not survive a blocking call that masks interrupts; TelemetryGuard was exactly that and never once fed during the http.POST() it existed to cover.
    • Never feed through a call you have not bounded. Feeding turns a recoverable watchdog reboot into a permanent freeze. A TLS handshake with no deadline wedged Core 0 with USB still enumerated and both the CLI and the web dead, until a hardware reset. Bound the call first, then feeding inside that bound is safe — that pairing is what tools/arduino_pico_overrides restores for BearSSL.

Reading the new metrics

show metrics → [STORAGE]:

Deliberately deferred (wave 3)

Resolved by wave 2 (2026-07-22)