You Don't Need an Agentic Framework to Start Building Agents
Stop researching frameworks and start building, the pattern is simpler than you think.
I’ve been having an absolute blast building AI agents lately. They are, without a doubt, one of the most exciting things in the entire AI stack. But it’s got me wondering: why isn’t every developer building them? Why are we still so stuck on AGI-hype and raw model debates when the real fun is right here, waiting to be built?
After downing way too much coffee and talking to a lot of agentic devs, I’ve realized what’s stopping people: It’s not the idea of agents, it’s the starting line.
As developers, we’re wired to find the “right” abstraction. This instinct, combined with a noisy market full of competing agentic frameworks, pushes us straight into the “Framework-First Trap.”
The result, unfortunately, is a learning curve that feels mismatched for a first-time build. We instinctively put on our “systems-builder” hat, which points us straight to a framework. This means we end up spending weeks climbing its steep nuances instead of spending an hour mastering the pattern’s simple fundamentals. It’s an understandable trap we get stuck on the framework’s opinions, not the pattern’s logic, which slows down getting that first, simple agent out the door.
Don’t get me wrong. Frameworks (like LangChain, Mastra, or CrewAI) are essential for scaling. I love each and every one of them for specific reasons. They absolutely save you a ton of time, give you production-grade plumbing, and have deep coverage for the hard stuff like memory, feedback loops, and advanced orchestration. (In fact, I love them so much I even wrote one myself: Not7.ai).
But they’re not the starting point. Too often, the first question is, “Which framework should I learn?” We immediately jump to the tool, assuming the tool is the pattern.
It’s not.
An agent isn’t a specific piece of software. It’s a design pattern. It’s a loop. Once you understand that loop, you realize all these tools are just different ways to express the same core idea.
An agent isn’t a specific piece of software. It’s a design pattern. It’s a loop.
And yes, it is absolutely fine to build your own agent from scratch and take it to production. As long as you consciously take care of the “stuff below”, especially the critical parts like secure tool handling and authentication (which we’ll cover in the checklist).
The Core Loop: How Agents “Think”
At their heart, all agents from the simplest chatbot to a complex multi-agent system run on a continuous loop, executing simple or complex steps towards one or more goals. You can call it whatever you want, but it boils down to this (a pattern inspired by papers like ReAct):
Reason: The agent looks at its goal (e.g., “answer the user’s question”) and its current state (what’s been said, what it knows). It “thinks” and forms a plan. Example: “The user is asking ‘what’s the weather in SF?’ I don’t know the weather. My plan is to use the
get_weathertool.”Act: The agent executes the plan. This usually means calling a tool (like an API, a database query, or a web search) or, if it has enough information, generating a final answer. Example:
CALL get_weather(location=”San Francisco”)Observe: The agent gets the result of its action. This is the new information. Example:
Tool returned: {”temp”: “65F”, “conditions”: “foggy”}Repeat: The agent takes this new observation, adds it to its memory, and starts the loop over. Example: “Okay, now I have the weather. My new plan is to give this answer to the user.”
That’s it. That’s the whole pattern.
Frameworks like LangChain just give you pre-built plumbing to manage this loop, package tools, and format memory.
No-code tools like OpenAI Agent Builder, N8N or Dify let you draw this loop on a visual canvas, connecting blocks that represent “Reason” (an LLM prompt), “Act” (an API call), and “Observe” (the API response).
A “DIY” agent is just a Python script with a
whileloop that calls an LLM, parses the JSON output, runs a function, and stuffs the result back into the prompt.
They are all the same loop.
At least for a start, for a simple agent that you want to build.. this itself will teach you so much on why? What’s missing? Which framework should I now choose? What is missing beyond the frameworks, like secure tool calling, auth, etc.?
Why You Should Probably Start with DIY
You can absolutely go live, start using your DIY agent, and get surprisingly far. Trust me.
Why? Because building it yourself forces you to skip the abstractions and immediately confront the problems that are actually more critical than your framework choice. You’re forced to get answers for:
Memory: “How do I really want to store the conversation? A simple list? A summarized version? A vector?”
Tools: “How do I tell the model what tools I have? How do I safely parse its output so it doesn’t try to
rm -rf /my server?”Guardrails: “What happens if the tool fails? What if the LLM gets stuck in a loop? How do I stop it?”
Orchestration: “What if I need two API calls? Does the agent know how to do that, or do I need to hard-code the logic?”
When you use a framework first, it answers these questions for you... with its own opinions. When you build it yourself, you find the answers while your agent is already running.
This DIY-first approach flips the script: you’re forced to focus on the stuff that actually matters from day one. You’ll know why you need a framework when the time comes, and you’ll know exactly what to look for. You’ll already be thinking about the right problems, the exact ones in this production checklist.
Anatomy of a Production-Grade Agent (The “Oh Crap” Checklist)
Going from a simple while loop to a production-ready agent is a huge leap. This is where frameworks can shine, but this checklist is the stuff you need to solve for any agent, DIY or not. These are the things that separate a “cool demo” from a “boring, reliable product.”
(We’ll do a deep dive on this list in a future post.)
Agent Scope & Design: What is this agent’s one job? What tools does it actually need to do that job? (Start with one!)
Guardrails & Cost Control: What stops the agent from getting stuck in a loop and bankrupting you on API calls? How do you filter for toxicity, PII, or prompt injections?
Observability & Logging: When it breaks (and it will), can you see the exact chain of thought? Can you replay the event to debug it?
Tool Management & Security: This is the part that gets gnarly. How are you passing API keys? How do you let the agent “act on behalf of a user” (e.g., “call the user’s own Google Calendar API”) without storing their credentials? This is a huge task—you’re building a whole auth/token management system. (This is a problem products like Arcade.dev are built to solve, letting you plug in secure, production-grade tool auth).
State Management (aka Memory): Is your memory system just a list in RAM , or is it backed by a real state store like Redis or Postgres or Zep?
Fallbacks & Human-in-the-Loop: What happens when the agent gets stuck? Does it just error out, or does it have a fallback (e.g., “I’m not sure, but here’s a web search”)? Can it flag the conversation for a human to review?
Evaluation: How do I even know if it’s “working”? You need a dataset of test questions to run against your agent to make sure its quality doesn’t drop when you make a change.
The Real Starting Line
Agentic frameworks are fine. No-code tools are great. But they are not the starting line.
The core of agent-building is just prompt engineering combined with a simple state machine—a glorified while loop, really. You don’t need a massive abstraction to start; you just need to understand how they “think.”
So open a text editor. Write that loop. Give it one tool.
Start small, build weird stuff, and learn the pattern. The agentic frameworks can wait.
What the first AI Agent you want to build ?