For years, AI product conversations centered on models: which one reasons better, which is cheaper, which has the largest context window. Those questions still matter, but shipping production systems with AI changed what I optimize for, because the model turned out to be one layer and the configured system around it turned out to be the product.
A capable LLM is a reasoning engine that produces language, code, summaries, classifications, and guesses about what to do next. Product behavior comes from everything wrapped around it, including instructions, tone, tools, memory, permissions, workflows, runtime, and security boundaries, and that whole system is the agent. Once agents act inside real workflows, they stop being experiments and become infrastructure.
The system is the agent
The first mistake many teams make is treating the LLM and the agent as the same thing. An LLM receives context and returns output, and it has no inherent knowledge of your product, codebase, customer, workflow, business rules, or security constraints, so you supply all of that around the model.
In Engineering in the AI Era I defined an agent as an LLM operating in a loop with tools. This post goes a level deeper, to the full shape you give that loop: a job, a role, a tone, capabilities, and boundaries, whether it is a coding agent, a support agent, a research agent, or a legal workflow assistant.
In a developer setup that shape usually comes from a system prompt, markdown instructions, a soul.md file, tool definitions, MCP servers, repo rules, and runtime code. In production it also needs storage, identity, access control, observability, retries, approval flows, queues, and security policies.
Prompt writing is one part of agent design, and each of the other parts governs something the prompt cannot: tools decide what the agent can do, memory decides what persists, permissions decide what it can reach, the workflow decides the order of steps, and the runtime decides where it lives and how it recovers from failure.
A basic agent may call one tool and return one answer, while a more advanced one follows a multi-step workflow, inspects documents, calls APIs, updates records, asks for approval, and coordinates with another agent. At that point you are shipping software with reasoning inside it rather than wrapping an LLM.
From agent implementation to agentic infrastructure
The smallest implementation is a tool-calling loop. The user asks for something, the model decides whether it needs a tool, the application executes the tool and returns the result, and the model either calls another tool or finishes. That pattern covers many cases with a model, a strong system prompt, a few typed tools, and a loop with a sane step limit.
Agentic systems outgrow a single loop quickly. Complex workflows need state across steps, persisted memory, retries, streamed progress, pauses for human review, the ability to resume after the browser closes, logs, and a clear user or organization identity, and they need safe actions separated from actions that require approval.
That is what I mean by agentic infrastructure: the layer that lets agents run reliably inside real products, covering runtime, orchestration, tools, security, observability, and ownership along with the model call.
Multi-agent setups add another dimension. When one agent owns research, another owns code changes, another owns QA or customer communication, and another owns compliance, you need orchestration that decides which agent acts, when control hands off, and how state survives the workflow.
That orchestration can be a few TypeScript functions, LangGraph, the OpenAI Agents SDK, a managed platform, or a custom service. The choice of tool matters less than the structure, because agentic systems need explicit boundaries instead of improvised loops.
Managed agents
Managed AI agents exist because most teams should not rebuild the same operational layer for every feature. A managed platform gives agents a place to run, remember, call tools, survive failures, and plug into product workflows without hand-wiring each piece of infrastructure.
Vercel approaches this from the web application side. The AI SDK gives TypeScript developers typed streaming, structured outputs, tool calling, UI integration, and agent abstractions that fit modern apps, so for teams on Next.js, React, or Node.js the agent can live next to the product interface instead of in a disconnected backend experiment.
Cloudflare approaches it from the infrastructure side. With Workers, Durable Objects, Workflows, and their Agents SDK, an agent behaves more like a stateful service with storage, scheduling, real-time connections, and lifecycle management, which matches how real agents work, since they need memory, schedules, tools, coordination, and a persistent connection to users or systems.
What a managed platform adds is a runtime. A chatbot can live inside a request, and a serious agent often cannot, because if it has to keep working after the user closes the browser, wait for an event, resume after a failure, call several tools, coordinate with another agent, or wait for human approval, it needs durable execution, state, and observability.
A platform supplies that runtime, and your team still defines the job, the tools, the boundaries, and the trust model. The platform runs the agent, and deciding what the agent may do inside your product stays with you.
Self-hosted agents
The opposite approach is owning more of the stack. Self-hosted agents give you control over where the agent runs, how state is stored, which models it uses, how tools are exposed, and how workflows are governed, which matters when the agent touches sensitive data, proprietary logic, internal systems, financial operations, legal workflows, or regulated infrastructure.
The tradeoff is operational load, since you manage runtime, persistence, retries, tool security, observability, model providers, memory, human-in-the-loop flows, testing, and production behavior yourself.
LangChain and LangGraph help here. LangGraph fits stateful, controllable, multi-actor workflows that need memory, human review, branching, and orchestration across steps or agents, and OpenAI's Agents SDK is another code-first path where your application owns orchestration, tool execution, state, approvals, custom storage, and integration with existing product logic.
Self-hosting is a choice to own more of the architecture, and for some teams that is the right call while for others it is weight they don't need.
Hybrid agents
Most serious AI products will end up hybrid. A team might use a managed model provider and own the tools, use the Vercel AI SDK for interface and streaming while keeping critical workflows in a private backend, run stateful agents on Cloudflare that call internal APIs elsewhere, orchestrate with LangGraph over managed APIs from OpenAI, Anthropic, or Google, or run self-hosted models for private workloads and managed models for general reasoning.
So the decision is which parts to manage and which to keep under your control. The model and the runtime can be managed while the workflow is custom, the tools are private, orchestration is code, memory lives in your database, the UI uses the AI SDK, a multi-agent graph runs on LangGraph, and execution uses workers, queues, durable objects, or whatever fits the product.
The right split depends on where risk and value live. Fast iteration on an AI-powered interface favors managed infrastructure, while proprietary workflows, sensitive data, internal tools, or regulated operations can justify more control, and most teams end up in the middle.
Building agents in TypeScript
Agentic systems do not require Python or LangChain by default. Python remains central for research, data tooling, evaluation, and model infrastructure, and LangChain and LangGraph matter for complex workflows, especially in Python-heavy environments.
Product engineers building web applications can stay in TypeScript to build agents. The Vercel AI SDK covers models, instructions, tools, structured outputs, streaming, approval flows, and UI integration inside apps that already have sessions, auth, billing, permissions, dashboards, databases, APIs, and background jobs.
If the product is already TypeScript and Next.js, keeping the agent layer close to the application layer is usually the practical choice. Deep data pipelines, model evaluation, and research-heavy systems may favor Python, and a product-facing agent that streams into a UI, calls typed tools, and uses existing APIs fits TypeScript well. What makes a system agentic is that it reasons, uses tools, follows workflows, preserves context, and acts inside constraints, in whatever language the product already uses.
The security boundary is the product boundary
The security model matters more as an agent becomes more capable. An agent with no tools is mostly conversational, and an agent with tools can read files, query databases, modify records, send messages, open pull requests, execute code, or call internal APIs, which is useful and carries a different risk profile.
Agent design becomes software architecture at that point, with questions about what the agent can access, which identity it uses, which actions are read-only and which are destructive, which need approval, what gets logged, what is reversible, and what happens when the model chooses badly, a tool returns bad data, or a user attempts prompt injection.
A prompt cannot answer those questions, and the system has to: narrow tools, clear permissions, strong input validation, human approval for risky actions, observability, and inspectable workflows. The aim is useful capability inside controlled boundaries, and once an agent can act, the infrastructure around it is part of the trust model.
Agent ownership inside teams
As agentic systems grow, ownership will split the way service ownership did in microservice architectures, with one team owning a research agent, another support, another coding, another the tools that expose internal systems, and another the orchestration across agents.
Agents also carry behavior, instructions, tone, memory, and decision boundaries that an API does not, so a team that owns an agent maintains its role in the organization: what it does, which tools it may use, how it behaves, how it fails, how it asks for help, and how it is evaluated.
This connects to The Forward Deployed Engineer. As AI embeds into customer workflows, the most valuable engineers understand the problem, shape the workflow, connect the product to technical systems, and ship what works in production, and agents amplify that role. A forward deployed engineer working with agents needs to understand the user, the workflow, the data, the tools, and the deployment environment, because an agent is useful when it fits how work actually happens.
Agents as a layer inside software
Agents don't replace applications, developers, workflows, or product thinking, and they add a new layer inside software. Some summarize, classify, route, or monitor, others create documents, prepare reports, update records, open pull requests, or coordinate tasks, and they may face users or run in the background on managed, self-hosted, or hybrid infrastructure.
What changes is that agents become things teams deploy, configure, observe, secure, and govern, the way they already treat infrastructure. As models get faster, cheaper, and easier to access, more product value moves to the agent layer of instructions, tools, workflows, memory, permissions, orchestration, and experience, and that is where engineering judgment goes.
When I design an agent I settle the same decisions each time: where it lives, what it can do, what it should remember, which tools it can call, when it asks for approval, how it recovers from failure, how it coordinates with other agents, and which parts are managed versus kept under the team's control. Those answers are the runtime around the model's behavior, and once the software can reason and act, they matter as much as the model does.