ADR-0006: Game Data Loading and Fail-Loud Validation — StreamingAssets JSON to Driftlands.Data, configVersion, TideConfig Arrival, the Validation Gate

Status

Accepted

Accepted 2026-08-28 — architecture-review-2026-08-28.md (verdict CONCERNS; review amendments applied before acceptance).

Date

2026-08-28

Engine Compatibility

ADR Dependencies

Context

Problem Statement

Every GDD ends with a load-time validation table and says "fail loudly", and the architecture's Principle 5 makes that a rule — but no document says where the JSON lives, how it reaches an engine-free assembly, in what order tables validate when rows cross-reference each other (a recipe's itemId must exist; a stationTier must be in Crafting's enum; harvestModifier ≤ 2.0 is Harvesting's ceiling on Crafting's data), what "fail loudly" does to the boot, or how configVersion and the tide's config epoch are derived. Three Proposed ADRs already consume the answer.

Constraints

• Driftlands.Data is engine-free (ADR-0003): it may hold records and validation rules, never file I/O or UnityEngine types.
• json_library (ADR-0004): Newtonsoft through explicit reads only; no reflection over gameplay types; deterministic behaviour under IL2CPP.
• Save Rule 4: unknown configVersion → refuse load, never migrate silently. Tide EC13/14: pool/window tuning applies at the next round boundary within a version; a socket-count change is a hard epoch — a mid-save content update the design explicitly allows.
• Coding standards: gameplay values are data-driven, never hardcoded; DI over singletons.
• Every validation row already specified by a GDD is binding as written (e.g. tide AC-36–43, combat AC-37–39, harvesting AC-32–36, crafting AC-40–47c, BB AC-55–64, enemy-ai density monotonicity, LDU sole-source warning, harvesting AC-25 no-home-nodes, movement/crafting/BB schema-negatives).
• Solo developer: tuning must be a text edit, not an asset pipeline.

Requirements

• One manifest + one JSON file per system under StreamingAssets/data/; readable and editable by hand.
• One boot gate: parse → build records → validate in dependency order → ValidationReport; any error halts boot on a named fatal-config screen; warnings log loudly and continue (the tide AC-42b / harvesting EC9 / LDU sole-source split).
• IGameData immutable after the gate; systems receive it by injection.
• configVersion from the manifest; socketCount recorded in the save's tide block; N change → AdvanceEpoch at load, before reconcile stage 4.
• Cross-system gates (supply ratio, bootstrap ratio, perimeter sessions, first-session ratio, harvest ceiling, strip-time readout) computed from live IGameData values both at boot (warning-level) and in CI (assert-level).
• Boot cost < 100 ms; zero runtime cost.

Decision

**Alternative A: JSON under StreamingAssets/data/ read synchronously at boot by a Runtime GameDataLoader through explicit Newtonsoft reads into engine-free *Def records; Driftlands.Data.Validation runs the GDD tables in dependency order into a ValidationReport; errors halt boot on a fatal-config screen; manifest.json carries configVersion; the save's tide block records socketCount and an N change calls AdvanceEpoch before reconcile stage 4.**

1. Files

StreamingAssets/data/
  manifest.json      { "configVersion": 1, "files": ["tide.json", "combat.json", …] }   ← configVersion bumped BY HAND (see §5)
  tide.json          TideConfig (ADR-0001)                       inventory.json  bagSlots, storageSlots, equipmentSlots, transferBatchWindow
  combat.json        knobs + archetype bands + unarmed 14        movement.json   moveSpeed, accelTime, remoteInterpBuffer
  harvesting.json    yields, bands, nodesPerLand, harvestModifierWarnCeiling, phase knobs   camera.json  followSmoothTime, aimBiasFrac
  input.json         moveStickDeadzone, aimStickDeadzone, aimPointerMinDistance, triggerPressPoint (ADR-0007 amendment; NO aim-persistence key — schema-negative)
  crafting.json      items (MaterialDef/GearDef + hooks + overrides + sourceTemplates), recipes, station tiers, durability, stationRange
  structures.json    StructureDef table, placementReach, socketNoBuildRadius, perimeterSegments   save.json  saveDebounceMs, backupCopies, flushDeadlineMs
  enemy-ai.json      archetype params, guardDensityByBand, aggro/leash/separation, telegraph cap
  land-dock.json     PreloadLeadTime (cited), bagFillSecs, nodeVisitSecs
  ui.json            uiScaleSteps, hudOpacityRange, reasonMinDisplayMs, reasonSwitchFrames, glintBandPx, pipQuotientMax + hud.md stage-9 rows (ADR-0014 amendment 2026-09-01; settings.json is runtime-written and deliberately NOT in the manifest)
  templates/<id>.json  TemplateDef: node counts per band, spawn points, socket layout (position, depositPoint, noBuildRadius), traversalProfile, addressableKey, candidateCells per band (ADR-0005 amendment; authored order load-bearing)

Desktop only: Application.streamingAssetsPath is a real directory on Windows/macOS/Linux; File.ReadAllText is used directly (no UnityWebRequest). The GDDs' assets/data/.json paths are these files: the repo folder assets/data/ is the source*, and Assets/StreamingAssets/data/ is a build-time copy (an IPreprocessBuildWithReport step mirrors the folder; the EditMode CI test reads the source folder via Path.Combine(Application.dataPath, "..", "assets/data") [SUPERSEDED — see Amendment 2026-09-03: the project lives in UnityProject/, so the source folder is Path.Combine(Application.dataPath, "..", "..", "assets", "data")]).

Shipped data is read-only; tuning overrides live in persistentDataPath (ruled 2026-08-28 — specialist B1). On macOS StreamingAssets sits inside the signed .app bundle: editing a file there breaks the code signature and Gatekeeper; on Windows a Steam install under Program Files may be read-only. So the bundle is never edited. If persistentDataPath/Driftlands/data-overrides/<file>.json exists for a manifest-listed file, it replaces that whole file (no key-level merging — merging is where silent half-edits hide); every active override is logged loudly at boot and shown on the main menu's debug line; validation runs identically over the resulting set. A friend tunes by dropping a file there, and a bug report can include it. Overrides never change configVersion (the manifest is canonical) — an override that needs a version bump is a content update, not a tune.

2. Loading (Runtime) → records (Data)

GameDataLoader (Driftlands.Runtime, plain class, no MonoBehaviour) runs first in the boot sequence (architecture §3.4 step 1) from a [RuntimeInitializeOnLoadMethod(RuntimeInitializeLoadType.SubsystemRegistration)] entry point — the earliest hook, before any scene object's Awake (so before a scene-placed NetworkManager can start), and the one Unity documents as re-firing every Play session under domain-reload-off, which is also where the previous session's GameData instance is discarded (no static state survives). Synchronously:

• Read manifest.json; a listed file that is missing is an error; an unlisted *.json in the folder is a warning naming it (TD-ADR C8 — a mis-named tuning file is still surfaced, but combat.json.bak from a tuning pass does not halt the boot).
• For each file: JObject.Parse → an explicit reader (TideConfigReader, CombatConfigReader, … in Driftlands.Data.Readers, taking an IDataReader seam identical in shape to ADR-0004's ISaveReader) → immutable records. Unknown keys and missing required keys are errors (a typo'd knob name must not silently fall back to a default — the "silent zero-effect weapon" class combat EC12 names). Every numeric knob's safe range from its GDD Tuning Knobs table is data too: readers accept the value, validators judge it.
• Assemble GameData : IGameData — Items, Recipes, Structures, Enemies, Templates, Configs.{Tide, Combat, Harvest, Craft, Build, Inventory, Movement, Camera, Input, Save, EnemyAI, Land, Ui} (Input added by ADR-0007; Ui added by ADR-0014), ConfigVersion, Manifest — all IReadOnlyDictionary/records.
• Validation.Run(GameData) → ValidationReport (§3).
• Report handling (§4). On success GameData is registered for injection and never mutated.

2a. Numeric width on config records (2026-09-07, review C7)

Every duration, distance, rate, fraction or modifier on a *Config record is double. float appears only where a Unity API demands it (shader/tint floats, ITideClock.TintFloats). Tide schedule math stays long/double per ADR-0001; ADR-0003 §2's "gameplay geometry is float" is about System.Numerics spatial values and grants no licence for float durations. Rationale (core-harvesting story-007): two configs feeding one state machine at different widths cost one ulp at exactly the boundary the machine is defined by — 0.10f widens to 0.100000001490116. HarvestConfig shipped as float before this rule existed and is retyped to double by a follow-up story, after which story-007's decimal-snap workaround is re-evaluated.

3. Validation — order and rows (Driftlands.Data.Validation, pure)

Validators run in dependency order; a stage runs only if every stage it references produced no errors (so a missing item does not cascade into a hundred "unregistered itemId" errors):

Each finding is ValidationFinding { Severity, Stage, File, Path, Rule, Message } — Rule cites the GDD row (e.g. combat.EC10, bb.AC-63b) so a failure message names its design source.

4. Failure behaviour

• Any error → boot halts before the first scene loads gameplay: a minimal fatal-config screen lists every finding with file/path/rule; the process stays open so the message can be read; NetworkManager never starts. The screen is an inline-styled UIDocument with no USS and no theme (6.3's strict USS parser must not be able to fail the screen that reports failures), with IMGUI OnGUI as the fallback if even that cannot render. Never a silent fallback to defaults (BB EC17's "structures.json fails to load on the async boot path" row is this rule).
• Warnings only → boot continues; each warning is logged at Warning with the same finding format; the count is shown once on the main menu's debug line. Warnings are the GDDs' "deliberately-surrendered invariants" (tide AC-42b, harvesting EC9 warn band, LDU sole-source).
• CI runs the same Validation.Run over the repo's assets/data/ in EditMode and fails the build on errors or on stage-10 gate breaches. Warnings are allowed — but their set is snapshotted (tests/fixtures/validation-warnings.snapshot): the build fails when the warning set changes without the snapshot being updated, so a deliberately-surrendered invariant (tide AC-42b, harvesting warn band, LDU sole-source) is acknowledged, not merely logged (TD-ADR C6). Stage-10 gates also carry BB AC-45b/46b's mutation-sensitivity shape: perturb one named constant by a known delta and assert the gate value moves by exactly the arithmetic-predicted amount.

5. configVersion and the config epoch

• configVersion is an explicit integer in manifest.json, bumped by hand for changes that alter derived state or the meaning of stored keys: StableHash64 salts, TemplateId renumbering, Def schema changes, save-key renames. It is not bumped for tuning: W/G/C, pool membership, N, yields, costs — those apply under the GDDs' own in-version rules (tide EC13 round-boundary pool change; EC14 hard epoch on N; everything else takes effect next boot). The save gate (ADR-0004 step 1) refuses a mismatch; the manifest is the sole source.
• Config epoch detection (tide EC14; ADR-0001 AdvanceEpoch's only caller): ADR-0004's tide block gains "socketCount": N — the N the save was last flushed under. Amendment to ADR-0004: §1 (the field) and §5 step 3 (which read "nothing to do" and now reads: epoch check — if Configs.Tide.N ≠ save.tide.socketCount → AdvanceEpoch; else nothing) — TD-ADR C2a. Ordering: Restore(TideState) → Advance(hostUtcNow) → epoch check (TD-ADR C2b): AdvanceEpoch(newConfig, activationT) takes activationT = clock.T after the monotonic guard — if the host clock is behind lastKnownGoodT (tide EC5), an epoch activated at raw loadUtc would sit in the clock's past and rebase dockingIndex/roundIndex before the current t. Different → epoch += 1, epochActivationT = clock.T, offsets recomputed, indices rebased (tide EC14); a loud "schedule shifted" notice is queued for the main menu; the next flush records the new socketCount. Equal → no epoch. Pool/W/G differences never trigger an epoch (EC13). A fresh world records socketCount at creation. ADR-0001 is unchanged: socketCount lives in the save's tide block, never in TideState — it is a load-time comparison input, not clock state; adding it to TideState would give the clock two sources for N (TD-ADR C2c).
• One ConfigVersion, one writer (TD-ADR C2d): ADR-0001's TideConfig.ConfigVersion is populated from manifest.configVersion by the tide reader — it is the same value, never authored in tide.json; the manifest is the sole source and IGameData.ConfigVersion is the canonical read.
• Load order (mechanising ADR-0004 steps 2–3): Restore(TideState) → Advance(hostUtcNow) → epoch check → reconcile stage 4 onward.

Architecture Diagram

 Boot (architecture §3.4 step 1)
 ────────────────────────────────
 StreamingAssets/data/manifest.json ─► GameDataLoader (Runtime, sync)
   ├─ tide.json … templates/*.json ─► JObject.Parse ─► *Reader (Data, explicit, IDataReader seam) ─► *Def / *Config records
   ▼
 GameData : IGameData (immutable)
   ▼
 Validation.Run (Data, pure)  stages 0→10 in dependency order ─► ValidationReport
   ├─ errors   ─► FATAL CONFIG SCREEN (no game data, no NetworkManager) — halt
   └─ warnings ─► log loudly; continue
   ▼
 inject IGameData → TideClock(seed, Configs.Tide) · HostContext.Data · ReconcileContext.Data · every system
   ▼
 Save.Load: Restore(TideState) ─► Advance(loadUtc) ─► if Configs.Tide.N ≠ save.tide.socketCount → AdvanceEpoch(newConfig, clock.T) ─► reconcile 4..9

 CI: Validation.Run(assets/data) in EditMode — errors OR stage-10 gate breaches fail the build

Key Interfaces

namespace Driftlands.Data
{
    public interface IGameData {
        int ConfigVersion { get; }
        IReadOnlyDictionary<ItemId, ItemDef> Items { get; }            // ItemDef: Material | Gear (hooks, overrides, durabilityPool), SourceTemplates: TemplateId[]
        IReadOnlyDictionary<RecipeId, RecipeDef> Recipes { get; }
        IReadOnlyDictionary<StructureId, StructureDef> Structures { get; }
        IReadOnlyDictionary<EnemyId, EnemyDef> Enemies { get; }
        IReadOnlyDictionary<TemplateId, TemplateDef> Templates { get; } // node counts per band, spawn points, SocketLayout, TraversalProfile, AddressableKey, CandidateCells (ADR-0005) — NO SourceMaterials: the mapping has one hand-authored home, ItemDef.SourceTemplates; a reverse index is derived at load if any consumer needs it (TD-ADR C7)
        GameConfigs Configs { get; }                                    // Tide, Combat, Harvest, Craft, Build, Inventory, Movement, Camera, Save, EnemyAI, Land — one record each
    }
    public interface IDataReader { /* same shape as ISaveReader (ADR-0004): TryInt/TryStr/TryBool/TryFloat, Array, Object — plus Keys() so unknown keys can be rejected */ }
    public enum Severity : byte { Error, Warning }
    public readonly record struct ValidationFinding(Severity Severity, int Stage, string File, string Path, string Rule, string Message);
    public sealed record ValidationReport(IReadOnlyList<ValidationFinding> Findings) { public bool HasErrors => …; }
    public static class Validation { public static ValidationReport Run(IGameData data); }    // pure; stages 0–10; CI and boot call the same method
}
namespace Driftlands.Runtime.Data
{
    public sealed class GameDataLoader {
        public static LoadOutcome Load(string dataRoot);                // LoadOutcome: Ok(IGameData, ValidationReport) | Fatal(ValidationReport)
        public static void ApplyConfigEpoch(TideClock clock, IGameData data, SavedTideBlock saved);   // the ONLY AdvanceEpoch caller; runs AFTER clock.Advance, uses clock.T as activationT
    }
}

Alternatives Considered

Alternative B: ScriptableObject assets per table

• Pros: Inspector editing, Unity-native references, validation attributes.
• Cons: Values live in YAML scenes/assets, not the JSON the GDDs name; validation over engine objects fights Driftlands.Data's purity (ADR-0003); an Editor-time validator cannot run in the CI EditMode suite without asset import; friends cannot tune a build by editing a text file.
• Rejection Reason: The GDDs already chose JSON paths and the architecture already chose engine-free validation.

Alternative C: Addressables core-group TextAssets

• Pros: consistent with land-template assets; async.
• Cons: the boot gate becomes an async ordering problem ("nothing else may start on failure" is harder to guarantee); a dependency the boot path does not need; hides the files inside bundles.
• Rejection Reason: StreamingAssets is synchronous, visible, and desktop-only is all we target.

Alternative D: Resources/ TextAssets

• Rejection Reason: Resources is discouraged in Unity 6 (always resident, no unload) and hides the files; no benefit over StreamingAssets on desktop.

Alternative E (versioning): content hash as configVersion

• Rejection Reason: every knob tweak would refuse every existing save — the opposite of the GDDs' "tuning applies at the next round boundary" rule.

Alternative F (epoch detection): treat any configVersion bump as an epoch

• Rejection Reason: the gate refuses a different configVersion, so the epoch could never actually happen at load; N must be detectable within a version — hence socketCount in the tide block.

Consequences

Positive

• One boot gate, one report format, one CI entry point; every GDD's "fail loudly" table has an executor and every failure names its design rule.
• Tuning is a text edit next to the executable; a friend's bug report can include the file.
• configVersion and the config epoch have defined derivations; tide EC13/EC14 are mechanised.
• Cross-system gates run from live values both at boot and in CI — the crafting-AC-54 lesson (no literals) applied everywhere.

Negative

• Explicit readers are more code than reflection deserialisation (one reader per file, ~12); the price of IL2CPP safety and stripping-proof loading under the json_library stance.
• Hand-bumped configVersion can be forgotten → mitigated by a CI test that fails when a Def schema or a hash salt changes without a bump (a snapshot of the reader set's shape).
• StreamingAssets is editable by players — friends-only, no anti-tamper (technical-preferences); a tampered file that fails validation halts their boot loudly, which is the desired behaviour.
• The fatal-config screen is a tiny bespoke UI outside the main HUD stack (it cannot depend on data that failed to load).

Risks

• Validation stages ordered wrongly → a cross-reference rule runs before its target table → spurious errors; mitigated by the stage table above and a test that the stage graph is acyclic and references only earlier stages.
• A GDD adds a validation row nobody mechanises → each GDD's validation table maps 1:1 to a validator class with the GDD row id in the Rule string; /architecture-review diffs the two.
• Boot cost grows with templates → Tier 1's 6–8 templates are still < 100 ms; measured in CI.
• A user edits the shipped bundle on macOS anyway → the app fails Gatekeeper, which is Apple's behaviour not ours; the override folder is the documented path and the main-menu debug line shows active overrides.
• Boot hook races a scene-placed NetworkManager → SubsystemRegistration precedes all scene Awakes (verified); the PlayMode boot test asserts IGameData is registered before NetworkManager initialises.

GDD Requirements Addressed

Performance Implications

• CPU: boot-only — parse + explicit reads + validation of ~12 files, target < 100 ms; zero runtime cost (immutable records).
• Memory: the records (tens of KB).
• Load Time: adds the gate to boot; measured in CI.
• Network: configVersion is one int in the late-join header (ADR-0008).

Migration Plan

No production code. (1) IDataReader + Newtonsoft implementation (shares ADR-0004's reader adapter); (2) manifest.json + tide.json with ADR-0001's validator as stage 1; (3) GameDataLoader.Load from the SubsystemRegistration hook + the override folder + the fatal-config screen; (4) CI EditMode test running Validation.Run over assets/data/ + the warning snapshot; (5) apply the ADR-0004 amendment (§1 socketCount, §5 step 3 epoch check) and update the registry's save_file stance in the same change; (6) one file + validator per system as its ADR lands; (7) stage 10 gates once harvesting/crafting/BB data exist.

Validation Criteria

• EditMode: each GDD validation row has a test that a violating file produces exactly that Rule finding at the right severity (error vs warning), and a clean file produces none.
• EditMode: stage ordering — a missing itemId produces one Items-stage error and no cascaded recipe/structure errors; the stage graph is acyclic.
• EditMode: unknown key / missing required key / unlisted file → error, never a default.
• EditMode: Validation.Run over the repo's assets/data/ yields zero errors and zero stage-10 breaches (the CI gate).
• EditMode: configVersion mismatch → save refused (ADR-0004); N differs from saved socketCount → AdvanceEpoch called exactly once before Advance; equal → never.
• Integration: a build with an intentionally broken combat.json shows the fatal-config screen naming combat.EC10, and NetworkManager never starts.
• /architecture-review shows TR-tide-009, TR-combat-012, TR-harv-006, TR-craft-001/005/007, TR-bb-001, TR-ai-005, TR-ldu-004 covered and QQ-08 closed.

Related Decisions

• ADR-0001 (TideConfig, TideConfigValidator, AdvanceEpoch), ADR-0002 (W for SoftCloseTick), ADR-0003 (Driftlands.Data, Schema_*), ADR-0004 (configVersion gate; amended: tide block gains socketCount)
• docs/architecture/architecture.md — Module Ownership (GameData), §3.4 init order step 1, Required ADR row 6, QQ-08
• Design: every GDD's "Config validation (load-time, fail loudly)" table; tide EC13/14; harvesting AC-28; crafting AC-54/56/57; BB AC-45/46; LDU Rules 3/4/7/8; enemy-ai Rules 6/9

Amendments

2026-09-03 — §1 source-folder path (foundation-game-data story-008)

§1 resolved the repo source folder as one level above Application.dataPath. That predates the decision to nest the Unity project under UnityProject/; from inside the project, assets/data/ is two levels up: Path.Combine(Application.dataPath, "..", "..", "assets", "data"). The single-.. form lands one level short and reads as a missing folder. The path is now resolved in exactly one place in the test assembly (SourceData.Root), guarded by one standalone Directory.Exists test that fails alone if the project moves; the Editor mirror (DataMirrorBuildStep.SourceRoot, story-007) uses the same two-level form. The CI gate (CiDataGateTests) runs GameDataLoader.Load — and therefore the one Validation.Run — over that folder; warnings are compared to tests/fixtures/validation-warnings.snapshot (stage|file|path|rule per line); stage 10 is asserted vacuous until Migration step 7 lands rows, so "zero breaches" is visibly not yet a claim.

2026-09-04 — stage 9 gains its first rows (foundation-input story-004)

input.json (already in §1's file list and row 9's title since ADR-0007's acceptance) is now read by InputConfigReader and validated by InputRules.Stage9 on the existing Knobs slot — stage 9 is no longer vacuous; the story-004 tripwire lists {2,3,5,6,7,8,10}. input.json is not yet in ManifestRules.RequiredFiles (an absent file is input.notLoaded at stage 9, fatal); foundation-input story-005 adds it when the adapter that needs it lands.

(Folded in 2026-09-08 from the stray docs/architecture/ADR-0006-game-data-loading.md, which had been written to the wrong filename — review U2, review-fixes story-008.)

Amendments — 2026-09-07 (architecture-review-2026-09-07.md)

• C7 new §2a pins numeric width on config records to double (no ADR pinned it; HarvestConfig float vs CombatConfig double cost a boundary ulp).
• C8 §5 diagram corrected: it showed the epoch check before Advance, contradicting §5's prose (TD-ADR C2b) and the shipped SaveLoader; TideClock throws on the wrong order.
• The stray docs/architecture/ADR-0006-game-data-loading.md (an orphaned 2026-09-04 stage-9 note) belongs in this section — fold and delete (review U2).