mtg/tools/embed.py
mark rosenberg d4a8a587f0 Restructure into a multi-deck engine; generalize tools/ off Jund/Reaper hardcodes
Migrates the reaper-king-no-more proof of concept into decks/reaper-king-no-more/
as the first deck under a shared tools/ engine, so the repo can hold many
Commander decks instead of one implicit one.

- tools/deckctx.py: new shared DeckContext, resolves --deck / $MTG_DECK / cwd,
  loads a deck's config.yaml, and derives color identity from the commander's
  own card data (never a hardcoded color combination).
- cards.py: drop the JUND=frozenset("BRG") constant; jund_legal() renamed to
  identity_legal(card, identity); DB() takes an explicit blacklist_path since
  the blacklist is per-deck, not shared.
- combos.py, roles.py, similar.py, similar_text.py, bracket_check.py: take
  --deck, use the deck's real identity instead of Jund/BRG.
- power.py, card_stats.py, forge_report.py: OURS deck-name constant now comes
  from the deck's config (name: / commander), matched via DeckContext instead
  of hardcoded "Reaper King No More".
- to_dck.py, batch_run.py, ab_test.py: .dck filename and [metadata] Name= are
  derived from the deck (slug / config name), not a hardcoded "reaper.dck".
- generate.py: RECIPES and make_variants.py's VARIANTS (deck-specific
  hypotheses) moved out of Python into decks/reaper-king-no-more/variants.yaml;
  protected cards now come from config.yaml's protected: list instead of a
  hardcoded set.
- New tools/mtg dispatcher: `tools/mtg decks/<name> <tool> [args...]` runs any
  tool against any deck without cd-ing into its directory.
- data/ (Scryfall/MTGJSON/Spellbook bulk, ~1.2GB, gitignored) and embeddings.npz
  move to the repo root as shared, commander-format-wide data.
- docs/howto.md: moved from the deck dir (it documents the tool, not the deck)
  and updated its example commands to the real generalized CLI.
- README.md: rewritten to describe the multi-deck repo and point at
  decks/reaper-king-no-more as the reference example.

Verified after the change: bracket_check, combos, and roles against
reaper-king-no-more reproduce the exact PASS/11-combos/role-profile numbers
from before the move; generate.py --list now loads recipes from YAML;
make_variants.py reproduces all 7 hand-curated variants including the
mikaeus FAIL control case; to_dck.py derives the right Name= and filename;
variant.py and compare.py reproduce decks/README.md's original numbers
exactly. The original POC at ~/mtg/reaper-king-no-more is untouched.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QPKgLYpMR8szWNFXyK1XNu
2026-09-12 00:37:42 -04:00

48 lines
2.2 KiB
Python

#!/usr/bin/env python3
"""Re-encode every card in the local Scryfall bulk file into mechanics embeddings.
Mirrors minimaxir/mtg-embeddings: same model, same "prettified JSON of the card's
mechanical fields" input, same unit-normalized output -- but built from OUR
oracle-cards.jsonl, so it is current and includes new commanders.
python3 tools/embed.py # -> data/embeddings.npz
"""
import json, os, sys, numpy as np
from sentence_transformers import SentenceTransformer
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
MODEL = "Alibaba-NLP/gte-modernbert-base"
SRC = os.path.join(ROOT, "data", "oracle-cards.jsonl")
OUT = os.path.join(ROOT, "data", "embeddings.npz")
def card_text(c):
"""The mechanical identity of a card, as prettified JSON. No flavor, no art."""
faces = c.get("card_faces") or []
# NOTE: name is deliberately excluded -- including it makes the model match on
# shared words in titles ("Reaper", "Massacre") instead of on mechanics.
d = {
"mana_cost": c.get("mana_cost") or " // ".join(f.get("mana_cost","") for f in faces),
"type": c.get("type_line"),
"text": c.get("oracle_text") or "\n//\n".join(f.get("oracle_text","") for f in faces),
"power": c.get("power"), "toughness": c.get("toughness"),
"loyalty": c.get("loyalty"),
}
return json.dumps({k: v for k, v in d.items() if v}, indent=2)
def main():
cards = [json.loads(l) for l in open(SRC)]
# skip tokens/art series -- they have no mechanics worth matching
cards = [c for c in cards if c.get("layout") not in ("token","double_faced_token","art_series","emblem")]
names = [c["name"] for c in cards]
texts = [card_text(c) for c in cards]
print(f"encoding {len(texts)} cards with {MODEL}", file=sys.stderr)
m = SentenceTransformer(MODEL, device="cuda", trust_remote_code=True)
emb = m.encode(texts, batch_size=64, normalize_embeddings=True,
show_progress_bar=True, convert_to_numpy=True).astype("float32")
np.savez_compressed(OUT, names=np.array(names, dtype=object), embedding=emb)
print(f"wrote {OUT} shape={emb.shape}", file=sys.stderr)
if __name__ == "__main__":
main()