# How to use this system A pipeline that generates thousands of legal Commander decks, plays them against each other in a real rules engine, and ranks **cards and card pairs** by how much they actually contribute to winning. It is not a deckbuilder that hands you "the best deck". It is a measuring instrument. What it produces is *evidence*; you still make the decisions. **This is the generic tool.** It works on any deck in `decks//` that supplies the four inputs below. Every command takes `--deck decks/` (or a positional deck path via `tools/mtg decks/ ...`), or falls back to `$MTG_DECK`, or to the current directory if you `cd` into a deck first. Nothing here is hardcoded to one commander or one color identity — see `decks/reaper-king-no-more/` for a worked example. --- ## What you supply Four things, in a deck's own `config.yaml`. Everything else is derived. ### 1. Your commander ```yaml commander: "The Reaper, King No More" ``` From this the system derives your **color identity** (which cards are legal at all), the legality filter, and which combos in the 108k-combo database are reachable. Color identity is *always* looked up from the commander's own card data — it is never a hardcoded color combination anywhere in `tools/`. ### 2. Your blacklist — cards you won't play `/blacklist.txt` (path configurable via `blacklist_file:`), one per line, with a reason: ``` Sol Ring | boycott, not a power-level call Talisman of Indulgence | prefer land ramp Mind Stone | prefer land ramp ``` Reasons are for you, not the tool. Blacklisted cards are excluded from the pool, from generation, and from "you're one card away" combo suggestions — so they never come back as a recommendation. ### 3. Your card pool The set of cards decks may be built from. Two ways to define it: ```yaml pool: max_edhrec_rank: 1000 # by popularity, or... file: mypool.txt # ...an explicit list you curate ``` **Pool size is the most consequential choice you make.** See *Choosing a pool size* below — it directly determines whether your results mean anything. ### 4. Your constraints ```yaml bracket: 3 # enforces game changers / MLD / extra turns house_rules: no_vacuum_combos: true # no 2-card infinites needing no board state lands: 36 ``` --- ## What you do NOT supply Derived from the card databases, no input needed: - **Color identity** — from the commander - **Which cards are legal** — Scryfall - **Game Changers list** — Scryfall's `is:gamechanger` (53 cards, auto-updates) - **What each card does** — 4,369 Scryfall Tagger tags, so the system knows what a sac outlet or a drain payoff *is* without you labelling anything - **Which card combos exist** — 108,535 combos from Commander Spellbook - **How popular a card is** — MTGJSON `edhrecRank` - **Opponents** — 173 Commander precons ship with Forge; 4 pods rotate automatically Optionally you can tune the **role budget** (how many sac outlets, removal spells, ramp pieces a deck should have) but sensible defaults are provided. --- ## Choosing a pool size Each deck has ~63 nonland slots. Coverage is what makes the statistics work: | pool | 100 decks | 500 decks | 1000 decks | | --- | --- | --- | --- | | 300 | 21 decks/card · 4.4x per pair | 105 · 21.8x | 210 · 43.5x | | **451** | 14 · 1.9x | **70 · 9.6x** | **140 · 19.2x** | | 978 | 6.4 · 0.4x | 32 · 2.0x | 64 · 4.1x | **Recommendation: 300–500 cards.** A 978-card pool at 1000 decks gives only 4x coverage per card *pair*, which is not enough to measure synergy. A smaller, curated pool measured well beats a large pool measured badly. Note the combinatorics: C(451, 63) is about **10^78** possible decks — comparable to the number of atoms in the observable universe. You are never searching this space. You are *sampling* it well enough to score individual cards. --- ## Choosing a game count Win rate in a 4-player pod has a 25% random baseline. Precision: | games per deck | 95% confidence interval | | --- | --- | | 30 | ±15.5pp — meaningless | | 50 | ±12.0pp — still noise | | 100 | ±8.5pp | | **150** | **±6.9pp** — usable | | 300 | ±4.9pp | **Use 150.** Below that you cannot distinguish a good deck from a lucky one. **But deck-level precision is not the point.** Card-level data pools across every deck a card appeared in: | card appears in | total games behind it | CI | | --- | --- | --- | | 14 decks × 150 | 2,100 | ±1.9pp | | 70 decks × 150 | 10,500 | ±0.8pp | | 140 decks × 150 | 21,000 | ±0.6pp | This is the real payoff. Individual decks stay noisy forever; **cards get sharp fast.** You are not running 1000 decks to find the best one — you are running 1000 decks as a sampling apparatus to score 451 cards. --- ## How long it takes Measured: ~48s/game on a modern desktop i7; expect ~120s/game on older Xeons. Forge is **CPU-only** — single-threaded per instance, no GPU involvement. Sizing: **1 worker ≈ 1 core ≈ 1.5 GB RAM.** | hardware | workers | limit | | --- | --- | --- | | 16 cores / 32 GB | 16 | cores (needs 27GB) | | 16 cores / 64 GB | 16 | still cores — extra RAM does not help | | 32 cores / 64 GB | 32 | cores (needs 51GB) | On a 16-core box at ~120s/game (~480 games/hour): | run | games | time | | --- | --- | --- | | 100 decks × 150 | 15,000 | **1.3 days** | | 500 decks × 150 | 75,000 | 6.5 days | | 1000 decks × 150 | 150,000 | 13 days | This is a job that *cooks*. The queue is persistent and resumable — kill it, reboot, resume, and you lose at most one deck's work. --- ## What you get out ### Card rankings For each card: how often it was cast, what turn it landed, and its win-rate contribution — computed by **ridge regression over deck composition**, not a raw average. Regression matters because good cards cluster together; it estimates each card's contribution *holding the rest of the deck constant*. ### Synergy pairs For cards A and B, compare the observed win rate of decks containing both against what you'd predict from their individual effects. A positive residual means the pair is worth more than the sum of its parts. This is what finds "these two belong together" rather than "both of these are good". ### Deck rankings Ranked list with confidence intervals. Use it to identify the top and bottom *deciles* — trust the tails, not the exact ordering. ### Never-cast list Cards that never got played across thousands of games. The most immediately actionable output: each one is dead weight, uncastable by the AI, or never drawn. All three are worth knowing. --- ## Tuning the AI (and why the defaults lie to you) Forge's AI is heuristic and its own docs say it is *"pretty bad for most combo decks."* Three fixable problems distort results before any deck question is asked: **1. The default clock silently discards your best games.** Forge's `-c` is a *per-game* wall clock defaulting to **120s**; on expiry the game is abandoned as a **Draw**. Nobody wins a draw. In the first 39 games logged here, **7 (18%) hit that limit** — and they are overwhelmingly the long grindy games an aristocrats deck is built to win. The default biases the measured win rate *downward*. `batch_run.py` now defaults to `--clock 600`. **2. Forge refuses to play 2,506 cards.** Cards carrying `AI:RemoveDeck:All` in their script are ones the engine will *never* cast. **7 cards in this deck** carry it, and **32 of the 451-card pool (7%)**. They appear as "never cast" in results, which reads as deck weakness but is a simulator artifact. `card_stats.py` now reports two lists: ``` NEVER CAST — AI-BLOCKED: 7 (simulator artifact, IGNORE) NEVER CAST — PLAYABLE: 7 (real signal, worth investigating) ``` They are blocked for real reasons, and the reasons differ: | card | why the AI can't cope | treatment | | --- | --- | --- | | Viscera Seer | bare sac outlet, **no AI logic written** | override (Woe Strider proves the pattern works) | | Goblin Bombardment | same, but the whole sac-for-damage class is blocked | override, tested separately | | Beast Within | gives an opponent a 3/3; can't weigh the downside | **substitute** | | Toxic Deluge | X-cost paid with *life*; no logic to size X | **substitute** | | Greater Good | draw X then discard 3; symmetrical drawback | **substitute** | *Overriding* only helps where nobody wrote logic. Where the AI genuinely can't evaluate a card, forcing it on makes it play the card badly — so `sim_substitutions.yaml` swaps in an AI-playable analogue **for simulation only**, keeping the deck's shape intact: ```yaml Beast Within: Feed the Swarm Toxic Deluge: Blasphemous Edict Greater Good: Umbral Collar Zealot ``` `decklist.txt` stays the truth; every swap is recorded in run metadata and printed. Note the tradeoff: per-card stats for a substituted card describe **the stand-in, not the real card**. This buys whole-deck realism at the cost of per-card precision. **3. `Default.ai` disables the sacrifice fallback entirely.** `SACRIFICE_DEFAULT_PREF_ENABLE` is **false** in stock Forge, so any sac outlet lacking a card-specific hint is simply never fed — fatal for this archetype. `forge-overrides/ai/Combo.ai` turns it on and widens the caps (`MAX_CREATURE_EVAL` 135 → 400, so the AI will feed real creatures rather than only tokens). ```bash python3 tools/batch_run.py --deck decks/ --ai-profile Combo --overlay --clock 600 ``` Our seat only — **opponents always stay `Default`**, or you just move the baseline. ## Measuring: power level, not win rate Win rate throws away everything that happened in games you lost, and needs 150 games to say anything. `tools/power.py` scores what the deck *did*: ``` DID THE DECK DO ITS THING? commander deployed 6/6 games (100%) median round 5 engine assembled 5/6 games ( 83%) median round 14 life taken from table mean 105.8 rounds survived mean 17.7 ``` A 17% win rate reads as failure; "commander down every game, engine online in 83%, 106 life drained off the table, half the games ran out of clock" is a deck that works. Use these to A/B a change — they move on far fewer games because they score *every* game. `tools/ab_test.py` runs two configs on identical decks and seeds and reports per-metric deltas with confidence intervals, labelling anything inside noise as `noise` rather than pretending it is a result. **Always run `--aa` first** — two identical arms must show no difference; if the null test reports `MOVED`, the harness is broken. ## The honest caveats **Forge's AI is heuristic and, by its own documentation, weak at combo decks.** It under-plays exactly the cards an aristocrats deck cares about. The tuning above mitigates this; it does not remove it. **A tuned AI is not stock Forge** — results are not comparable with untuned runs, which is why every run records its config. Everything here measures *what Forge's AI can pilot*, which is a proxy for your pod, not a substitute. Cards that need a human to see the line will score badly. **Opponents are stock precons** (roughly bracket 2). Your results describe performance against that field. **The AI cannot do politics.** In a real pod, threat assessment and table talk decide games. Forge plays a free-for-all somewhat like three parallel duels. **Correlation, not causation.** A card cast in long games looks good because long games are ones you survived. Regression helps; it does not eliminate this. Use the output to generate hypotheses, then test them with a variant. Do not let a win-rate column pick your deck. --- ## Running it ```bash # one-time setup on a new machine (data/ is shared across every deck in the repo) python3 tools/fetch_data.py # ~1.2GB of card data python3 tools/embed.py # -> data/embeddings.npz (needs GPU or ~20min CPU) # install Forge 2.0.14 to ~/forge (needs Java 17+) # configure a deck $EDITOR decks//config.yaml # commander, pool, constraints, roles $EDITOR decks//blacklist.txt # cards you won't play $EDITOR decks//decklist.txt # the 100 cards # check it, offline and instant python3 tools/bracket_check.py --deck decks/ python3 tools/combos.py --deck decks/ python3 tools/roles.py --deck decks/ # generate role-budget variants (hypotheses live in decks//variants.yaml) python3 tools/generate.py --deck decks/ --list python3 tools/generate.py --deck decks/ # or hand-curated cut/add variants (variants.yaml `handwritten:` section) python3 tools/make_variants.py --deck decks/ # run the full analysis suite for one variant, optionally with Forge games python3 tools/variant.py --deck decks/ no-mikaeus # analysis only, ~10s python3 tools/variant.py --deck decks/ no-mikaeus --sim 24 # + 24 Forge games, ~25 min # batch-simulate every variant in the deck in parallel python3 tools/batch_run.py --deck decks/ --games 150 --workers 16 --heap 2g python3 tools/compare.py --deck decks/ # rebuild decks//decks/README.md # same thing through the single dispatcher, without cd-ing into the deck tools/mtg decks/ bracket_check tools/mtg decks/ variant no-mikaeus --sim 24 ``` Everything lands in `decks//decks//` — decklist, legality check, combo analysis, per-card stats, and every simulation run as `.log` / `.txt` / `.json`. See the top-level `README.md` for the full repo layout.