① Intuition
General-purpose agents struggle in open-ended environments like Minecraft because they face two compounding problems. First, tasks have long horizons: surviving and thriving requires hundreds of interdependent steps — chop wood, craft tools, mine stone, smelt iron — none of which can be solved with a single prompt. Second, every skill is potentially useful again: if an agent learns to mine coal in episode 3, it shouldn’t re-derive that procedure from scratch in episode 47. Prior embodied agents either hard-coded a fixed skill set (brittle) or used raw action traces (non-reusable when the world changes). The result: agents that plateau early and never compound their knowledge.
Voyager’s insight is to treat a growing skill library as the agent’s long-term memory and to write skills as executable code rather than recorded action sequences. Code generalizes: mineWood(count=10) works regardless of the agent’s exact starting position, whereas a trace of button presses replays incorrectly whenever the world layout differs. Voyager builds three mutually reinforcing modules on GPT-4: (1) an automatic curriculum that proposes progressively harder tasks, (2) a skill library that stores verified JavaScript/Mineflayer code snippets keyed by natural-language descriptions and retrieved by embedding similarity, and (3) an iterative prompting loop that writes, executes, receives environment feedback, and self-improves the code until it passes.
Before touching the widget: if you slide the episode counter from 0 to 4, you expect to see 4 skills unlock in sequence. Which skill do you predict unlocks last — `craftFurnace`, `mineCoal`, or `mineStone`? And why does the dependency graph matter for that answer?
craftFurnace and mineCoal both unlock at episode 4 and 5, but mineStone unlocks at episode 3 because it is a prerequisite for both — you need stone tools to mine efficiently. The dependency graph matters because skills are not independent: smeltIron requires craftFurnace AND mineCoal AND mineIron, all of which require mineStone. The library grows as a directed acyclic graph, not a flat list, and the curriculum must respect this topological order.
Interact with the widget: slide the episode counter from 1 to 8 and watch skills unlock one by one. Then toggle “Show dependencies” to reveal the dependency graph. Notice how smeltIron has three incoming edges — it is gated on three earlier skills all appearing in the library. This is the structure that emerges naturally when skills are stored as composable code with explicit function calls.
Voyager — Skill Library
library size = 0
- ○mineWood(ep 1)
- ○craftPickaxe(ep 2)
- ○mineStone(ep 3)
- ○craftFurnace(ep 4)
- ○mineCoal(ep 5)
- ○mineIron(ep 6)
- ○smeltIron(ep 7)
- ○craftIronSword(ep 8)
After playing with the widget: at episode 6 the library contains 6 skills. If a new task arrives that requires `smeltIron`, which stored skills would the retrieval system likely surface, and why would using code snippets (rather than action traces) make those retrieved skills directly reusable?
The retrieval system would surface mineIron, craftFurnace, and mineCoal because their natural-language descriptions are semantically close to “smelt iron ore.” Code snippets are directly callable: smeltIron() can internally invoke craftFurnace() and mineCoal() by name, and those functions still work regardless of the agent’s current map coordinates. An action trace (e.g., “move 3 blocks north, break block at 2,64,5”) is position-dependent and breaks whenever the world is regenerated — which happens constantly in Minecraft.
Retrieval check
Answer all 2 correctly to mark this stage mastered.
② Math
The skill library M stores entries of the form (d_i, c_i, e_i) where d_i is a natural-language description, c_i is the executable code, and e_i = \text{embed}(d_i) is the embedding vector. Retrieval for a query description q selects the top-k entries by cosine similarity:
The automatic curriculum generates the next task t* by maximizing expected exploration conditioned on full agent state:
where state encodes current inventory and nearby blocks, C_{\text{done}} is the set of completed tasks, and C_{\text{fail}} is the set of recently failed tasks. This is not an explicit optimization — the LLM acts as a learned heuristic that has absorbed exploration principles during pretraining.
Show next line
Show next line
Retrieval check
Answer all 2 correctly to mark this stage mastered.
③ Pseudo code
Arrange the five steps of Voyager’s main episode loop in the correct order:
- Mark task complete or failed; update completed_tasks / failed_tasks; advance to next episode
- Retrieve top-k skills from library: context_skills = top_k(embed(t), M)
- If task succeeded: add new skill to library — M.add(description, code, embed(description))
- Curriculum proposes next task t = LLM_curriculum(state, completed_tasks, failed_tasks)
- Iteratively prompt GPT-4 to write/debug code with execution feedback until verified or retry limit hit
Fill in the blank — the line that inserts a newly verified skill into the library:
def add_skill(library, description, code):
embedding = embed(description)
library.append(_______)
return libraryRetrieval check
Answer all 1 correctly to mark this stage mastered.
④ Code (Python)
Implement a minimal skill-library retrieval in NumPy. Given a list of skill dicts (each with "description" and "embedding" as a np.ndarray), a query_embedding (np.ndarray), and an integer k, return the top-k skills sorted by descending cosine similarity to the query. If the library has fewer than k entries, return all of them.
Retrieval check
Answer all 1 correctly to mark this stage mastered.
⑤ Original paper
Suggested reading order: §1 (Introduction — motivation from hard-coded vs. open-ended agents) → §3 (Method — overview of the three modules) → §3.3 (Skill Library — storage, retrieval, and verification details).