How we built AI villagers that learn and unlearn without retraining the model

At each decision, the world supplies the actions a villager can physically take. A small model called Julia-1 scores those options and helps choose among them. It does not invent new actions or write a full plan.

Each villager also has personal memories. These contain things the character did, saw or was told, rather than everything that happened in the world. EmbeddingGemma helps find the memories that relate to the current situation. This lets a past experience become part of a new decision.

For selected uncertain or consequential choices, a larger model called Gemma takes a turn. It can consider the villager’s situation and the available actions before a decision is made. A failed plan or a blocked route can make this extra thought useful, even when the quick model was confident about its first choice.

Julia-1's weights stay fixed in this experiment. What changes is the information and preferences around it: remembered experiences, written rules and numerical scores that make some actions more or less likely. These scores are called action priors. They give past outcomes a way to influence later choices without retraining the model.

This division of work lets a routine choice remain quick while preserving a way to consider a more difficult situation. The slower model does not need to write every action for every villager. Memory supplies context from that particular character’s experience, and the fast model still works within the actions offered by the world.

A useful experience can become a short rule such as when to act and what to avoid. The system stores each rule with its owner, a confidence score and a history of outcomes. Each villager can carry up to five rules. A self-discovered rule starts at confidence 1, while a borrowed rule starts at 0.5.

When a relevant situation returns, the rule can appear in the context for a decision. Action priors also track which strategies have worked. The wording of the available choices matters too: in early tests, explaining a river crossing's food yield more clearly changed how willing Julia was to cross. A later choice can therefore reflect several parts of the system working together.

For example, a villager that learns a safer way around an obstacle can carry that lesson into a later encounter. Confidence gives the system a way to treat the lesson as provisional. A rule that has helped repeatedly can remain available, while an outcome that contradicts it can reduce its influence over time.

The villager’s diary provides another view of this process. It is written from that character’s memory, which may be incomplete. Reading the diary alongside a stored rule helps explain why a choice made sense to that villager at the time.

A villager can learn after witnessing another's successful action. To decide whether an event is visible, the simulation casts 25 sight lines through terrain and obstacles. At least 42% must reach the target, and rain and darkness affect visibility. This is a geometric visibility check rather than an image recognition model.

A trusted conversation can also pass on a rule. Receiving a rule does not ensure that it will be used: borrowed rules rarely appeared explicitly in later decisions in the observed runs.

Rules can lose confidence when they lead to poor outcomes. A failed application lowers confidence by 0.25, while a successful one raises it. A rule is removed when its confidence reaches zero. In one run, Arun's confidence in a direct-confrontation rule fell from 1.0 to 0.25 after three unsuccessful attempts. The rule weakened in that example, but had not yet disappeared.

We compared three conditions across 15 seeded valleys: fast decisions alone, memory with slow deliberation, and the full learning system. Each valley faced a scheduled flood and a later rockfall. The main question compared the full learning system with memory alone on the amount of food returned during the rockfall.

The learning condition returned an average of 4.80 loads per valley, compared with 4.13 for memory alone. This difference did not meet the study's significance criterion (p = 0.073). A secondary result showed 8 rockfall injuries with learning, compared with 41 with memory alone.

The villagers' choices help explain the injury difference. Those with memory alone chose to climb over rocks 63 times, while those with learning chose it 18 times and took more detours. In this comparison, they avoided more of the dangerous action rather than becoming resistant to injury.

These results describe two different parts of village life. Bringing food home asks whether the village can keep doing useful work after the path is blocked. Injury counts show whether the villagers keep choosing a dangerous route. Learning changed the second behavior more clearly in this experiment.

The system offers a way to follow a learning loop: an experience can produce a rule, a rule can influence a later choice, and the outcome can strengthen or weaken it. Memories and confidence scores make parts of that process visible.

The valley comparison gives a specific result, rather than establishing that this approach always learns better. The next article takes up a different question: how to share limited computing capacity among many villagers.