Generative Agents: Interactive Simulacra of Human Behavior

Park and colleagues' UIST 2023 paper gives each of 25 simulated townspeople a memory stream of timestamped natural-language records, retrieves from it by the sum of min-max-normalized recency, importance and relevance scores, writes reflections and plans back into the same stream, and reports that 100 raters ranked the full architecture's interview answers as the most believable (TrueSkill 29.89 against 21.21 with no memory), with each removed component lowering the rating.

Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, Michael S. Bernstein (Stanford University; Google Research; Google DeepMind) · 2023-04-07 · original

Can a language model play one character consistently over days of simulated time, when everything the character has seen will not fit in a prompt? This paper answers with an architecture built around a single store, the memory stream: a growing list of natural-language records of what an agent observed, concluded and planned. A retrieval function picks the records to show the model at each step by adding three scores (how recent, how important, how relevant), and two further processes, reflection and planning, write the model's own syntheses back into the stream. Twenty-five such agents, run on gpt-3.5-turbo, lived for two game days in a small sandbox town. In a controlled test, 100 raters ranked interview answers from the full architecture as more believable than answers from ablations that removed planning, reflection or all memory, and more believable than answers written by crowdworkers role-playing the same agents (Section 6.5, Figure 8). The evaluation measures believability as judged by people, not task success, and the paper's own error analysis finds retrieval failures and embellished memories in the full architecture.

Version of record. The paper appeared in the Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST '23, San Francisco, 29 October to 1 November 2023), 22 pages, DOI 10.1145/3586183.3606763. It was first posted to arXiv (2304.03442) on 7 April 2023, the date given above. This note follows arXiv v2 (6 August 2023), which carries the UIST header and DOI and is the camera-ready text, read together with its LaTeX source; the ACM Digital Library copy was not opened. Section and figure numbers are v2's. The authors released the simulation code on GitHub (joonspk-research/generative_agents); the note's remarks on the code refer to commit fe05a71 of 11 August 2023, the latest commit when read.

Terms used in this note

  • Generative agent. The paper's term for "computational software agents that simulate believable human behavior" (abstract). A footnote states that phrases like "the agent goes to the cafe" are shorthand and "not a suggestion that they are engaging in human-like agency" (Section 1, footnote 1). This note keeps that reading.
  • Memory stream. The paper's long-term store: "a list of memory objects", each holding "a natural language description, a creation timestamp, and a most recent access timestamp" (Section 4.1).
  • Observation, reflection, plan. The three kinds of record in the stream. An observation is "an event directly perceived by an agent"; a reflection is a "higher-level, more abstract" thought generated from other records; a plan is a future sequence of actions with location, start time and duration (Sections 4.1 to 4.3).
  • Believability. The dependent variable of the evaluation, taken from earlier work on believable agents (Section 6.1). It is a judgment by human raters, not a correctness score.

How the memory works

What is stored, and when it is written

Everything is text. Records enter the stream at four moments:

  • At initialization. Each agent starts from one paragraph written by the authors, describing occupation and relationships; "each semicolon-delimited phrase" becomes a separate seed memory (Section 3.1).
  • At every time step. The sandbox server sends each agent the agents and objects within a preset visual range, and these perceptions are stored as observations (Sections 4.3.1 and 5). Observations include the agent's own actions, other agents' actions and object states, such as "The refrigerator is empty" (Section 4.1).
  • When reflecting. Reflections are written periodically (below), with pointers to the records cited as evidence (Section 4.2).
  • When planning. Day plans and their decompositions are saved to the stream (Section 4.3).

Each record also receives an importance score when it is created: the language model is asked to rate "the likely poignancy" of the memory on a scale of 1 to 10, where 1 is "purely mundane" and 10 "extremely poignant" (Section 4.1). The paper reports that the prompt returns 2 for "cleaning up the room" and 8 for "asking your crush out on a date". In the paper's description no record's text is ever edited or deleted (only its last-access time changes); the stream only grows.

How it is retrieved and scored

Retrieval takes the agent's current situation as a query and scores every record in the stream on three components (Section 4.1):

  • Recency: "an exponential decay function over the number of sandbox game hours since the memory was last retrieved", with decay factor 0.995. In symbols (our notation), a record last accessed $h$ game hours ago has raw recency $0.995^{h}$. Because the clock is reset by access, retrieving a record makes it more likely to be retrieved again.
  • Importance: the integer rating fixed at creation.
  • Relevance: the cosine similarity between an embedding of the record's text and an embedding of the query. The paper says only that it uses "the language model" to generate the embeddings; the released code calls OpenAI's text-embedding-ada-002.

Each component is rescaled to $[0, 1]$ by min-max scaling over the candidate records, and the score is their weighted sum, written in the paper as

$$score = \alpha_{recency} \cdot recency + \alpha_{importance} \cdot importance + \alpha_{relevance} \cdot relevance,$$

with every $\alpha$ set to 1 in the authors' implementation (Section 4.1). "The top-ranked memories that fit within the language model's context window are included in the prompt." The paper gives no evaluation of the retrieval function by itself, and it names tuning these functions as future work (Section 8.2).

The query differs by use. To decide how to react to an observation, the agent runs two retrievals, "What is [observer]'s relationship with the [observed entity]?" and "[Observed entity] is [action status of the observed entity]", and the model summarizes the results (Section 4.3.1). In dialogue, the speaker retrieves memories about the other agent and about the last utterance (Section 4.3.2). In reflection, the queries are questions the model generated (below).

What enters the context

The model never sees the stream itself. Prompts are assembled from (Sections 4.3, 4.3.1 and Appendix A):

  • an agent summary description, a cached paragraph built from three retrievals ("[name]'s core characteristics", "[name]'s current daily occupation" and "[name's] feeling about his recent progress in life"), each summarized by the model, concatenated with name, age and traits, and regenerated "at regular intervals" (Appendix A);
  • the current date, time and status;
  • the observation at hand;
  • a model-written summary of the retrieved records ("Summary of relevant context from John's memory: ...");
  • for dialogue, the conversation so far.

So retrieved records usually reach the acting prompt through a summarization step, not verbatim.

Reflection

Reflection exists because, with observations alone, an agent asked whom it would like to spend an hour with picked the person it saw most often, not the one it shared interests with (Section 4.2). A reflection is triggered "when the sum of the importance scores for the latest events perceived by the agents exceeds a threshold (150 in our implementation)", which in practice happened "roughly two or three times a day" (Section 4.2). The process has three steps:

  1. The 100 most recent records are given to the model with the prompt "Given only the information above, what are 3 most salient high-level questions we can answer about the subjects in the statements?"
  2. Each generated question is used as a retrieval query; the retrieved records, which can include earlier reflections, are numbered.
  3. The model is asked "What 5 high-level insights can you infer from the above statements? (example format: insight (because of 1, 5, 3))", and each insight is stored as a reflection with pointers to the records it cites.

Because reflections can cite reflections, the result is a tree whose leaves are observations (Figure 7). Reflections are retrieved like any other record; there is no separate store for them.

Planning and reacting

Without plans, the paper reports, an agent asked what to do at each moment ate lunch at 12, 12:30 and 1 pm (Section 4.3). Planning is top-down: the model writes a day plan of five to eight chunks from the agent summary and a summary of the previous day, then recursively decomposes it into hour-long chunks and then into 5 to 15 minute actions, all saved to the stream. In practice only the high-level plan is generated in advance, and the near future is decomposed "just in time" (Appendix A). At each step the model decides whether to continue the plan or react; a reaction regenerates the plan from that time, and an interaction between agents starts a dialogue generated turn by turn from each agent's retrieved memories (Sections 4.3.1 and 4.3.2).

The world is also given to the model as text: the town is a tree of areas and objects, each agent keeps its own partial, possibly out-of-date subtree, and the model chooses a location by descending that tree one level per prompt (Section 5.1).

The released code

This section is the note's reading of the code at commit fe05a71 (still the head of the repository in November 2026), not a claim of the paper. File and function names are those of reverie/backend_server/persona/.

  • Weights. The scored retrieval, new_retrieve in cognitive_modules/retrieve.py, multiplies the three normalized components by hard-coded weights gw = [0.5, 3, 2], applied to recency, relevance and importance in that order, with commented-out alternatives [1, 1, 1] and [1, 2, 1] and a comment that "[1, 1, 1] tends to work decently". (These multiply per-agent weights recency_w, relevance_w and importance_w, which are 1 in every released starting file.)
  • How many records. new_retrieve returns a fixed number of top records per query, not as many as fit the context: 30 by default (used by planning and reflection), and 50, 25 or 15 in the conversation code (cognitive_modules/converse.py).
  • Recency clock. extract_recency computes recency from each record's position in last-access order, not from elapsed game hours: new_retrieve sorts all records by last_accessed in ascending order (least recently accessed first), and extract_recency assigns the $i$-th record in that list the value $\text{decay}^{i}$ for $i = 1, \dots, n$.
  • Recency direction. That assignment gives the largest raw value to the least recently accessed record. With three records accessed at times 1, 2 and 3 and decay 0.995, the raw values are 0.995, 0.990 and 0.985 for the oldest, middle and newest, and min-max scaling (normalize_dict_floats) turns them into 1, about 0.50 and 0; running the two functions as released (commit fe05a71) on these three records gives 1.0, 0.4987 and 0.0. As written, the code therefore ranks the least recently accessed record highest on recency, the reverse of the paper's description ("a higher score to memory objects that were recently accessed"). This is the note's reading and run of the code; whether the runs reported in the paper used this code is not stated.
  • Decay factor and threshold. The 25 starting files of the 25-agent town (base_the_ville_n25) set the decay factor to 0.995 and the reflection threshold (importance_trigger_max) to 250, where the paper says 150. The three-agent starting files (base_the_ville_isabella_maria_klaus) use 150, with decay 0.995 for one agent and 0.99 for the other two; 0.99 and 150 are also the defaults in memory_structures/scratch.py.
  • Reflection input. generate_focal_points in cognitive_modules/reflect.py gives the model the records perceived since the last reflection (counted by importance_ele_n), not a fixed 100 most recent records. The threshold counter is decremented by the importance of perceived events in cognitive_modules/perceive.py.
  • Two retrieval paths. The step that decides how to react to what an agent perceives calls a different function, retrieve, which gathers stored events and thoughts that share a subject, predicate or object keyword with the perceived event, with no scoring. The scored new_retrieve is used for planning, reflection and conversation.

The paper does not say which settings or code version produced its reported runs, so these differences show that the published description and the released code are not the same; they do not show which one generated the paper's results.

What the paper measures

Controlled evaluation: interviews ranked by believability

Setup (Sections 6.1 to 6.4). Agents were taken from the end of a two-game-day simulation run with the full architecture. Each was asked 25 questions, five in each of five categories: self-knowledge, memory, plans, reactions and reflections (listed in Appendix B). One hundred raters recruited on Prolific, paid 15 US dollars an hour, each watched a replay of a randomly chosen agent's life, with access to all of its memory stream, and saw one random question from each category with the answers produced under five conditions. They ranked the five answers from most to least believable (a within-subjects design).

What was varied (Section 6.2). The full architecture against three ablations, which removed access at interview time to (1) reflections, (2) reflections and plans, or (3) everything in the memory stream; and a human condition, in which one crowdworker per agent watched the replay, inspected the memory stream and wrote answers in the agent's voice. The authors describe the fully ablated condition as representing the earlier state of the art for language-model agents. All ablations answered from the memories the full architecture had accumulated; the authors chose this because re-simulating each condition would make the towns diverge, and they argue the differences therefore "likely represent a conservative estimate of the true differences" (Section 6.2).

Analysis. The rankings were converted to TrueSkill ratings, a Bayesian generalization of Elo ratings (compare Rating Models from Pairwise Comparisons); significance was tested with a Kruskal-Wallis test on the raw ranks and Dunn post-hoc tests with Holm-Bonferroni correction (Section 6.4).

Results (Section 6.5.1, Figure 8):

Condition TrueSkill $\mu$ $\sigma$
Full architecture 29.89 0.72
No reflection 26.88 0.69
No reflection, no planning 25.64 0.68
Crowdworker-written 22.95 0.69
No observation, no reflection, no planning 21.21 0.70

The Kruskal-Wallis test gave $H(4) = 150.29$, $p < 0.001$; all pairwise differences were significant at $p < 0.001$ except between the crowdworker condition and the fully ablated one, the two lowest. The paper reports a standardized effect size of $d = 8.16$ ("eight standard deviations") between the full and fully ablated architectures, obtained from TrueSkill's normal model of each condition's skill, $N(\mu, \sigma^2)$, "through Cohen's d". The paper does not give the formula, and the printed values do not reproduce it: the difference in means is $29.89 - 21.21 = 8.68$, which gives $d \approx 12.2$ when divided by the pooled $\sigma$ ($\sqrt{(0.72^2 + 0.70^2)/2} \approx 0.71$) and $d \approx 8.6$ when divided by $\sqrt{0.72^2 + 0.70^2} \approx 1.00$ (the note's arithmetic). That $\sigma$ is the uncertainty of a condition's rating, not the spread of raters' judgments, so "eight standard deviations" is not comparable to a Cohen's $d$ computed between groups of raters; this reading is the note's. Two further points are the note's own: each rater ranked all five conditions, so the five groups of ranks are not independent, which the Kruskal-Wallis test assumes; and the paper reports no agreement statistic between raters.

Qualitative findings (Sections 6.5.2 and 6.5.3, from the first author's inductive coding). With memory, agents recalled acquaintances and events that the ablated agent denied knowing. Failures remained: an agent who had heard about a candidacy said he had not been following the election, and another retrieved a plan to discuss the election at a party but not the memory of the party itself. The authors found it "rare" for agents to fabricate knowledge completely, but agents embellished: one added that a candidate would make "an announcement tomorrow", and another described a neighbor named Adam Smith as the author of Wealth of Nations, knowledge from the language model rather than the town. With reflections, an agent could name a suitable birthday present for a friend; without them she said she did not know what he liked.

End-to-end evaluation: two days of a 25-agent town

One simulation of 25 agents over two game days, measured by interviewing every agent at the end (Section 7.1):

  • Information diffusion. Agents who knew of one agent's mayoral candidacy rose from one (4%) to eight (32%); agents who knew of a Valentine's Day party rose from one (4%) to thirteen (52%). The authors checked each "yes" against the memory stream and found none hallucinated (Section 7.1.2).
  • Relationships. Network density of the mutual-acquaintance graph, $\eta = 2|E|/(|V|(|V|-1))$, rose from 0.167 to 0.74. Of 453 answers about knowing other agents, 1.3% (6) were hallucinated.
  • Coordination. Five of the twelve invited agents came to the party. Of the seven who did not, three cited conflicts when interviewed and four said they were interested but had not planned to come.

The scenario was seeded by the authors (the party host's intent and, for one guest, a crush on another agent; Section 3.4.3). The paper reports one run and no repeats.

Errors observed (Section 7.2): as agents learned more locations they chose atypical ones (lunch at the bar); physical norms not stated in the text were broken (several agents in a one-person bathroom, entering shops after closing); and the authors attribute overly formal speech and over-agreeable behavior to instruction tuning of the underlying model, to the point that one agent's interests drifted toward others' suggestions.

The instructions used

The architecture is a set of prompts, and the paper prints the main ones:

  • the importance prompt (1 to 10 poignancy rating, with brushing teeth and a break-up as anchors; Section 4.1);
  • the two reflection prompts quoted above (Section 4.2);
  • the day-plan prompt, an unfinished numbered list for the model to complete (Section 4.3);
  • the reaction and dialogue prompts, each built from the agent summary, the time, the status, the observation and a retrieved-context summary (Sections 4.3.1, 4.3.2);
  • the location prompt, which includes the instruction "Prefer to stay in the current area if the activity can be done there" (Section 5.1);
  • the agent-summary prompt (Appendix A).

A user can speak to an agent as an outside persona or as the agent's "inner voice", which "makes the agent more likely to treat the statement as a directive" (Section 3.1.2). The paper does not test alternative wordings of any prompt, and the weights $\alpha$ were not varied in the evaluation.

Limitations and open questions

  • What is measured. Believability rankings of single interview answers, by raters who could see the memory stream. Whether the memory retrieved was correct is assessed only qualitatively, and no retrieval metric is reported.
  • Ablation at interview time. Ablated conditions answered from the full architecture's memories, so the experiment varies what the model may read, not what it would have experienced.
  • Human baseline. One crowdworker per agent, not experts; the authors say it does not represent maximal human performance (Section 8.2), and it was statistically indistinguishable from the no-memory condition.
  • One model and one run. All results use gpt-3.5-turbo (GPT-4's API was invitation-only at the time; Section 4) and one two-day simulation. The authors name varying models and hyperparameters as future work.
  • Cost. Simulating 25 agents for two days was "costing thousands of dollars in token credits and taking multiple days to complete" (Section 8.2).
  • Robustness. The authors list prompt hacking, "memory hacking" (a conversation that convinces an agent of a past event that never happened) and hallucination as untested (Section 8.2).
  • Code and paper differ in the retrieval weights, the number of records returned, the recency clock and its direction, the reflection threshold and the reflection input (above). A reimplementation should follow one or the other deliberately.

Research questions

  1. How much does each retrieval component contribute? Ablating recency, importance and relevance one at a time, with a retrieval measure such as recall of the records a human marks as needed for each interview question, would separate the components that the believability study tests only together.
  2. Do the paper's equal weights or the code's weights retrieve better? Running the same interview set under $\alpha = (1,1,1)$ and the released $(0.5, 2, 3)$ for recency, importance and relevance, scored as in question 1, would answer it; the comparison should also fix the direction of the recency term, which the released code inverts (above).
  3. Do the ablation gaps survive re-simulation? Running each ablated architecture through its own two-day simulation, with several seeds, would test the authors' argument that interview-time ablation understates the differences.
  4. How often are reflections wrong, and do wrong reflections persist? Labeling a sample of reflections against their cited evidence, and tracking how often wrong ones are retrieved later, would measure error propagation through the reflection tree.
  5. Does the believability ranking track factual accuracy? Scoring the interview answers for correctness against the memory stream and correlating with rater rankings would show whether raters reward accurate memory or fluent embellishment.

How it connects

References

  • Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., and Bernstein, M. S. (2023). Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST '23), 22 pages. https://doi.org/10.1145/3586183.3606763. arXiv:2304.03442 (v2 read, with its LaTeX source). record · 2 pages
  • Park, J. S., et al. (2023). generative_agents (code), commit fe05a71d3e4ed7d10bf68aa4eda6dd995ec070f4, 11 August 2023. https://github.com/joonspk-research/generative_agents (files reverie/backend_server/persona/cognitive_modules/retrieve.py, reverie/backend_server/persona/memory_structures/scratch.py and the agents' bootstrap_memory/scratch.json read). record
  • Herbrich, R., Minka, T., and Graepel, T. (2006). TrueSkill: A Bayesian skill rating system. In Advances in Neural Information Processing Systems 19. (Cited by the paper for its rating method; not read for this note.) record