Java's Agentic Framework Boom is a Code Smell
Stop building orchestration frameworks and start building agents, the future belongs to those who master the ecosystem, not those trapped building language specific engines
Last week, I published a post arguing that you don’t need an agentic framework to start building agents. That piece was for the developers waiting on the sidelines for the “perfect” framework to arrive.
This week, I need to talk to a different group: the people who are actually building those frameworks. I know this group well because I was one of them.
I need to come clean. I’m a framework-aholic. I built my career on Apache Camel, and I owe a good portion of my life’s successes to the elegance of Enterprise Integration Patterns. I get it. And if there’s one community that deserves the Nobel Prize for Frameworks, it’s the Java community. From the early days at Red Hat to the entire big data ecosystem, frameworks have been the engine of the JVM world for 15 years. We are the masters of abstraction.
So when the agentic era dawned and Java was playing catch-up, my first instinct was primal: build a framework. I even started one, driven by the thought, “Where is the Apache Camel for AI agents?”
Three months ago, there was maybe one serious Java agentic framework. Now, there are three, including Embabel. The race is on. But watching this explosion, I’m forced to ask a difficult question: Is the framework itself becoming an anti-pattern? Are we creating a liability for ourselves instead of focusing on what really matters: building agents
The recent boom in Java agentic frameworks isn’t a sign of a healthy, maturing ecosystem. It’s a symptom. A code smell at the architectural level that tells us something is fundamentally wrong with our approach.
Why Did We Build Frameworks in the First Place?
Let’s rewind. Why did frameworks like Spring and Camel become so dominant? The reasons were clear and valid:
Developer Productivity: We were drowning in boilerplate code. Frameworks abstracted it away.
Code Quality & Governance: They provided standardized patterns that prevented every developer from reinventing the wheel.
Reusability: They gave us battle-tested constructs to build upon, saving immense time and effort.
The goal was to optimize for productivity, quality, and governance. But are those the same parameters we should be optimizing for today? It feels like we’re trying to solve a 2010 problem in 2025, completely ignoring the elephant in the room: AI-powered developer tools.
The Elephant Has a Name: Cursor (and Friends)
Here’s what changed while we were busy porting LangChain to Java:
Cursor and Copilot can generate boilerplate faster than you can type an import statement. That elaborate chain abstraction you’re building? Cursor writes it in three seconds. That tool registration pattern you’re standardizing? Copilot already knows five variations.
But here’s where we need to stop and ask a more fundamental question: What does your end user actually need?
What Do You Really Need to Build?
Let’s get specific. There are two scenarios most of us face:
Scenario 1: You’re building one critical agent over the next 12 months. Maybe it’s a customer service agent handling 10,000 conversations a day. Or a code review agent that needs to understand your company’s specific standards. Or a compliance agent that must never hallucinate on regulatory requirements.
Scenario 2: You’re building an agent platform. Hundreds or thousands of agents, each with different contexts, different domains, different requirements. Maybe you’re at a consultancy building agents for multiple clients. Or you’re creating an internal platform where different teams can spin up their own agents. You need something reusable, adaptable, evolvable. Something that lets you create new agents quickly while maintaining consistency and quality across all of them.
In both cases, ask yourself honestly: Does your user need a code framework?
Or do they need something completely different?
Redefining the Framework
Here’s what I’ve learned after abandoning my framework and actually shipping agents: We don’t need to eliminate frameworks. We need to redefine what a framework actually means in the AI era.
The old framework definition: A reusable code abstraction that provides structure and handles cross-cutting concerns. Something you import and build on top of.
The new framework definition: The complete environment for building agents, a set of interdependent layers that work together, where the code layer is just one piece of a larger puzzle.
Here are the layers that actually matter in a modern agent framework:
Layer 1: The Language Itself
Java (or your language of choice) with its constructs, types, and patterns. Not wrapped in abstractions, used directly. The language is already your framework for structuring logic. You don’t need a code framework on top of Java. Java IS the framework.
Layer 2: The Model
A really good LLM: GPT-5, Claude, Gemini, Grok. This isn’t just an API you call. It’s a core component of your stack. The model’s capabilities directly determine what you can build. Choose it as carefully as you’d choose your programming language.
Layer 3: Developer Productivity Tools
Cursor, Copilot, and the next generation of AI-powered dev tools. These are NOT optional. They’re critical infrastructure. Your framework should be designed to work seamlessly with these tools. If Cursor can’t easily generate code in your pattern, your pattern is wrong or you probably have to describe your pattern well.
Layer 4: Prompt Packs & Guidelines
Your versioned, tested, governed prompts. Your organizational voice. Your domain knowledge. Your compliance rules. This is where your business logic lives not in code, but in carefully curated context and instructions. Think of these as your dependency artifacts, like JARs, but for agent behavior.
Layer 5: Ecosystem APIs
Context awareness about the emerging ecosystem of specialized platforms and their APIs. Vector databases for knowledge retrieval. Context stores and memory management systems like Zep or Cognee. Tool execution platforms like Arcade. Observability platforms for agent monitoring like Langfuse. Prompt management and versioning tools. Most of these expose REST APIs or standard protocols, most of these provide LLM.txt for context import. Your framework needs awareness that these exist and knowledge of how to connect to them.
Layer 6: Architecture & Design Patterns
Architectural thinking captured as guidelines and patterns. Reusable blueprints for how these layers compose together across different use cases. Not code abstractions - documented knowledge on routing logic, versioning strategies, deployment patterns, and ecosystem integration that becomes part of your organizational context.
Think about it. When you build that critical customer service agent, what actually determines its success?
The Java code that calls the LLM? (That’s 20 lines, Cursor writes it)
The elaborate chain orchestration? (Standard control flow)
The retry logic and error handling? (Java already has patterns for this)
Or is it:
The model you chose and how well it handles your domain
The prompts that teach it your escalation rules and voice
The tooling that lets you iterate rapidly on those prompts
The integrations with platforms like Arcade for tools and Zep for memory
The architecture that lets you version, test, and deploy changes
The design patterns that let you reuse this approach across agents
That’s your framework. All six layers, working together.
The Framework in Practice
Let me show you what this actually looks like when you’re building agents:
Layer 4 (Prompt Packs) are versioned artifacts, not strings in your code:
prompts/
customer-service/
v1.2/
system-prompt.md
escalation-rules.md
tone-guidelines.md
product-context.md
examples/
refund-scenarios.yaml
technical-issues.yamlLayer 5 (Ecosystem APIs) configuration in your environment:
Your ecosystem context is embedded in guidelines:
# Ecosystem Integration Guidelines
## Tool Discovery
- Call Arcade API to list available tools: GET /tools
- Reference: See Arcade LLM.txt at https://docs.arcade.dev/llms.txt
## Memory Management
- Zep session API: https://api.getzep.com/v2/sessions/{session_id}
- Reference: See Zep API docs at https://docs.getzep.com
## Infra & Storage
- Object storage for prompt artifacts: S3, GCS, or Azure Blob
- State persistence for long-running workflowsLayer 1 (Java) provides the structure, clean and simple:
public class CustomerServiceAgent {
private final Model model;
private final PromptPack prompts;
private final ArcadeClient tools;
private final ZepMemory memory;
public Response handle(CustomerQuery query) {
// Retrieve conversation memory
var history = memory.getSession(query.sessionId());
// Get available tools from Arcade
var availableTools = tools.listTools();
// Render prompt with context
var context = prompts.render(”main”, query, history, availableTools);
return model.complete(context);
}
}Layer 3 (Cursor) generates this code in seconds. You focus on the architecture.
Layer 6 (Architecture) guidelines:
# Agent Architecture Guidelines
## Workflow Routing
- Design routing logic for multi-node agent workflows
- Classification node → routes to specialist nodes (support, sales, technical)
- Complexity analysis → routes to appropriate model tier (GPT-4o vs GPT-3.5)
- Tool selection node → routes to tool execution nodes based on user intent
- Route tool execution through Arcade gateway: centralized auth, rate limiting, tool discovery
- A/B routing for prompt versions: 10% to v2.0, 90% to v1.5, monitor quality
## Rate Limiting & Throttling
- Per-user token budgets: 10K tokens/day (free), 100K (paid)
- Queue management: Max 100 concurrent requests, overflow to SQS...
..
..Why This Framework Scales
For one critical agent: Pick your model (Layer 2), write clean code (Layer 1), iterate with Cursor (Layer 3), refine prompts (Layer 4), integrate ecosystem APIs (Layer 5), follow architectural patterns (Layer 6).
For a thousand agents: Same model, same architectural patterns, same ecosystem APIs but each agent gets its own prompt pack. Cursor generates the boilerplate. Your language provides structure. The ecosystem handles the hard problems.
The beauty? The layers work together. Cursor generates code because patterns are simple. Prompts are independently versioned. Integrations use REST APIs. Architecture enables reuse without abstraction.
No orchestration framework needed. This IS the framework.
The Engine vs. SDK Question
Let me be clear: I’m not saying all frameworks should disappear. I have tremendous respect for what the teams at LangChain, LangGraph, Mastra, CrewAI, Autogen and others have built. But we need to understand a critical distinction that’s getting lost in the rush to port everything to Java.
Don’t confuse an engine with an SDK.
Here’s what I mean: I can’t wait to develop full agents in Java. I love Java. But I don’t want a Java engine just because I want to develop agents in Java.
Consider these examples:
LangChain4J? Great start as an SDK that connects to the broader LangChain ecosystem. You write in Java, but you’re leveraging a proven engine.
Crew AI with a Java SDK? Perfect. Master the orchestration patterns in Python, then give me a Java interface to use them.
Mastra with multi-language support? Exactly the right direction. Build the engine once, provide SDKs for every language.
Add Java SDK or add any language SDK to Not7 which was built using Go ?
The pattern here? Develop in your preferred language without rebuilding the entire engine in that language.
The Orchestration Layer is Getting Thinner
Here’s why I think even the SDK approach might be temporary, or at least becoming far more minimal:
On one side: Models are getting dramatically better. GPT-5, Claude 4.5, and Gemini 2.5 Pro, Grok have reasoning capabilities that make complex orchestration patterns obsolete. They can handle multi-step workflows with simple prompts that would have required elaborate chains six months ago.
On the other side: The real engineering problems are being solved by specialized platforms. Take Arcade as an example tool discovery, authentication, execution at scale, handling rate limits, managing tool versions. That’s where the hard engineering work is. Tool management isn’t an orchestration problem anymore; it’s an infrastructure problem being solved at the platform layer.
In the middle: Orchestration frameworks are being squeezed thinner and thinner.
When your model can reason through workflows and platforms handle the complex engineering problems (tools, memory, context), what’s left for orchestration?
The answer: Very little. And that’s why the engineering focus needs to shift from orchestration to the broader agent development challenges prompt management, ecosystem integration, tool decision auditability, cost optimization. The real problems aren’t in orchestration anymore.
The New Reality: AI-Native Frameworks
The code smell isn’t just that we’re building too many frameworks. It’s that we’re building frameworks for a world that no longer exists. Here’s what building frameworks in 2025 actually means:
1. Accept the AI-powered development reality Every developer building agents will use Cursor, Copilot, or similar tools. That’s not a trend it’s the new baseline. Design your framework to work seamlessly with AI code generation, not against it. If Cursor can’t understand your patterns, your patterns are wrong.
2. Your framework is plain English, not just code The most critical parts of your framework will be well-engineered prompts, clear guidelines, and structured context not clever abstractions. These are your versioned artifacts. These determine agent behavior. Treat them with the same rigor you treat code.
3. Don’t reinvent engines when you need SDKs Yes, Java SDKs are critical. But you don’t need to rebuild entire orchestration engines just to write agents in Java. The ecosystem already has platforms solving the hard problems: memory (Zep, Mem0), tools (MCPs, Arcade), vectors (Weaviate, Pinecone, Qdrant), observability, etc. Integrate, don’t rebuild.
4. Frameworks are still critical but not for orchestration If you’re solving real problems prompt versioning, decision auditability, ecosystem integration patterns, cost optimization build that. But orchestration? The ecosystem has moved on. Memory, tools, context, observability are being solved by specialized platforms. Focus your innovation elsewhere.
5. Trust your language If you feel there’s a framework missing in your language of choice, step back. Modern languages Java, Python, TypeScript, Go are incredibly powerful. With their latest features plus AI code generation tools, you can build sophisticated agents with clean, simple code. Your language isn’t the limitation trying to wrap it in unnecessary abstractions is.
The framework of the future isn’t a code library you import. It’s mastery of six interdependent layers: your language, your model, your dev tools, your prompts, your ecosystem integrations, and your architecture.
The framework of the future isn’t a code library you import. It’s mastery of six interdependent layers: your language, your model, your dev tools, your prompts, your ecosystem integrations, and your architecture.