Composing an Agentic Research System

A stack becomes a research system only when its parts have jobs, boundaries and a shared record.

Start from practice, not the stack

Names such as model, agent, skill, MCP server, CLI and plugin describe surfaces. They do not yet describe a research arrangement. A list of what is installed leaves the consequential choices invisible: who owns the question, what evidence the AI can reach, what survives a handoff, and who may approve the next action.

This page works the other way round. It begins with seven distinctions you can notice in your own practice without any technical vocabulary, whether the work is fieldwork, experiments, datasets, archives, theory, models or writing. It then gives the five questions that locate any arrangement, and only after that names the surfaces. None of the distinctions is a level to reach. Different tasks in one project can sit on different sides of each, and a conversation is often the right arrangement.

Seven distinctions

Answering or acting

A proposed operation is different from one the system performs. An assistant that writes an R script and hands it back is answering; you run it, read the error, paste it back. A tool-using agent runs the script itself, reads the error, and tries again, while you set the boundary and judge the analysis. The second arrangement removes the copy-and-paste loop. It does not make the statistical claim any better supported, and passing code tests establishes nothing about the estimate.

The qualitative version is the same shape. An assistant that suggests codes for a transcript is answering. An agent that opens every transcript in a folder, applies the codebook and writes a memo per file is acting. Interest in the second arrangement is not evidence of using it, and a coherent set of memos is not evidence of fidelity to the material.

Who chooses the next step

When the route is unclear, one of three things happens. You design the workflow and the AI executes it. The AI proposes a workflow and you revise it before it runs. Or you state the goal and the boundaries and delegate the construction of the workflow itself. These are control modes, not maturity stages. The further construction is delegated, the more you must be able to read the plan, the evidence rules and the decision rights back from artifacts rather than from memory.

Chat or project

Ask what would survive if the conversation vanished now. In a chat, the answer is usually nothing: the reasoning, the intermediate results and the decisions live in a scrollback that will not be opened again. In a project, the AI works inside a folder whose files, records and decisions outlast any single session, and can be read by a collaborator or by a different AI. A literature scan that logs its queries and raw returns to files is a project; one that summarises "the main papers" in a reply is a chat, however good the summary.

Improvised or reusable

If you find yourself explaining the same procedure every time, that procedure can be written down once. A written procedure an agent can load is a skill: how to search, what to check, when to stop. A skill is not the thing it describes. It can tell an agent how to run a scan; it cannot retrieve a record. Retrieval needs a tool, reached through a connector (an MCP server), a command-line utility or an API. The two fail differently. A weak skill needs better procedure; a missing field needs a different tool.

Several opinions or different jobs

Running the same prompt through three models produces three opinions. It does not produce independent evidence, because the three share the framing, the input and the omissions. A differentiated team gives each agent a different job, a different slice of the evidence, or a different responsibility: one searches declared terms, one tests adjacent vocabulary, one reads only the raw records and challenges the frame. Agreement among differentiated agents means something. Agreement among copies means less than it looks.

For interpretive work the distinction is sharper still. A design that rewards convergence will make ambiguous accounts look coherent. A design that assigns someone to keep the tension alive protects what the material actually said.

On demand or recurring

Some work starts when you ask. Some starts on a schedule, or when a folder changes, or when a monitored source updates. Scheduling, initiative and authority are separate things: a recurring job can still require your approval before it acts on anything, and a one-off job can be given wide latitude. Ask what starts the next round and who authorised it.

Carried in your head or in shared records

How does a collaborator, or a second AI application, know what changed and why? If the answer is "I tell them", the context is human-carried, and it will not survive a holiday or a handover. If the answer is "it is in the project record", the decisions, tensions and changes of mind are somewhere readable. Durable records of reasoning are different from durable records of sources: a reference library preserves what was found; a research memo preserves why the question moved.

Five arrangements

The distinctions above combine into a short vocabulary of arrangements. It is a map, not a ladder.

1. Conversation: you ask, it answers, and you carry everything between turns.
2. Tool-using assistant: it can run code, read files or search, and checks with you as it goes.
3. Continuing project agent: it works inside a project whose files, decisions and records outlast the chat.
4. Differentiated team: several agents with different jobs on different evidence.
5. Shared project across hosts: more than one AI application working from the same records.

Repeatability, initiative, continuity and oversight vary inside each of these. A differentiated team can require approval at every consequential step; a single agent can run a scheduled task unattended. None of the five changes who owns the research question.

Five functional questions

Whatever the arrangement, five questions locate it. They travel across products, and they reveal when an arrangement is more elaborate than the task requires.

Each failure has a different remedy. A weak instruction needs better procedure. A missing capability needs a different tool. A lost decision needs durable state. An unauthorised action needs a stronger gate.

What each surface contributes

Only now do the technical names earn their place. The table classifies surfaces and functions, not brands; one project can expose several rows at once.

Memex, for example, is packaged as a plugin and a CLI, while its durable records serve as state. A reference manager such as Zotero is a platform, a source library and, through its MCP server, a set of callable operations. A useful system rarely needs every row.

The smallest useful system

Before adding a component, ask what observed problem it solves.

• Use one careful reader when the material is small and interpretive continuity matters more than parallelism.
• Use a deterministic script when the operation has a stable rule and no interpretive discretion.
• Add an agent when a bounded role benefits from contextual reasoning and leaves an inspectable artifact.
• Add another agent when it receives a meaningfully different brief, evidence slice or responsibility.
• Add durable memory when a decision or a tension must survive the session.
• Add an enforced gate when a recurring failure cannot safely depend on recollection.

Complexity is earned by a better evidence trail, a protected tension or a safer decision. It is not earned by the number of boxes in the architecture. Direct human work, one model, a script or a spreadsheet remain valid designs.

The consequence gradient

Controls should strengthen as the system moves outward from your own notes.

The gradient is why the same architecture needs different gates in different research. A system that helps produce the account it will later analyse, as OpenInterviewer does, sits further along it than one that reads existing material.

Where to go next

@misc{docs-advanced-topics-composing-agentic-research-systems-2026,
  author = {Xule Lin},
  title = {Composing an Agentic Research System},
  year = {2026},
  howpublished = {\url{https://research-memex.org/docs/advanced-topics/composing-agentic-research-systems}},
  note = {ORCID: 0000-0001-7885-4194}
}