Spring AI vs calling the model yourself: when the framework is worth it

I have done both in the same company. One service talks to a model through Spring AI. Another talks to the same vendor through a 120-line client we wrote because we already had an internal gateway and a single use case. Both shipped. Only one of those choices would I repeat in each place. The question is not “is Spring AI good.” The question is when the framework is cheaper than the call you already know how to make.

If you want the feature list, I already wrote what Spring AI actually buys you. This is the fork: framework bean versus your own HTTP.

What “calling it yourself” means here

I do not mean assembling JSON with a string builder in a controller. I mean a small client: the vendor SDK or a RestClient, timeouts you set, a record for the response, a test that stubs the wire. Sometimes it means talking to an internal LLM gateway that already does auth, routing, and spend caps. That gateway is the framework. Adding Spring AI in front of it is a second hat.

Calling it yourself is not morally cleaner. It is fewer moving parts when you only have one part.

When the framework is worth it

You are already a Spring Boot shop and the feature will not stay a single method. Second use case, third model, a tool, a bit of retrieval. The ChatClient and the config style pay for themselves when the call sites would otherwise fork. I have watched two teams invent two slightly different OpenAI wrappers. We then spent a quarter making them rhyme. Spring AI is that rhyme if you start there.

You want the call to look like the rest of the platform. Same metrics names, same actuator story, same @ConfigurationProperties, same test slice. That consistency is a real on-call gift. A one-off client gets forgotten in the runbook. A bean next to the datasource does not.

You will use tool calling or structured output in more than one place. Doing that well is not hard, but doing it twice is how you get two JSON schemas and two ways to fail. The framework’s version is good enough that I will not reimplement it for sport.

You do not have a gateway yet, and you are not about to build one for a single team. Spring AI is a local standard. A company gateway is a company standard. Do not pretend they are the same size of project.

When calling it yourself is worth it

One endpoint, one vendor, one prompt, no tools. A thin client is a morning. Spring AI is a dependency you will now explain to every new hire who wonders why chat is a framework. I do not add a framework to feel official.

You already have a gateway. Talk to the gateway. The gateway owns keys, routing, and the bill. Spring AI talking to the vendor around the gateway is how you get a shadow spend and a security review you already passed once.

The model or protocol is awkward in the starters. Weird streaming, a local model with a custom API, a partner who only speaks a half-documented RPC. I would rather a 150-line adapter we own than a fight with an abstraction that is 80% right. Eighty percent right is a second bug tracker.

The team is not a Spring team. Do not add Boot to get Spring AI. I have heard that proposal. It was a diagram proposal, not a staffing proposal.

You need to pin behavior harder than the abstraction wants. If you are going to unwrap every call anyway to log a custom trace or to enforce a tenant header the advisors do not know about, you are already writing a client. Write it on purpose.

The cost people forget

Framework cost is not download size. It is the upgrade, the surprise default, the advisor you did not know was on, the model swap that looks safe in yaml and is not safe in behavior. Own-client cost is duplication and the day someone copies the client and forgets the timeout. I pick the cost I can see. In a large Spring estate, I can see the estate. In a three-class service, I can see the three classes.

Time-to-first-demo favors Spring AI if the team already knows Boot. Time-to-first-honest-production favors whichever path already has timeouts, a flag, and a budget. I have shipped the demo path and then bolted those on. I would rather the thin client with the flag than the framework without one. I would rather the framework with the flag than a thin client that grows a second brain in week six.

A rule I actually use

If I can name two call sites or two models in the next six months, I start with Spring AI in a Boot app. If I can only name one, I start with a client or the gateway. If I am unsure, I start with the client and I do not pretend I will “just swap later.” Swapping later is a rewrite of the prompt and the tests, not a property change, no matter which path I picked.

If the company is about to bless a gateway, I wait a sprint and use the gateway. Being two weeks late is cheaper than being the unofficial second path.

If the feature is extract-and-fill or a human-in-the-loop draft, either path works. I pick the one the on-call already knows. If the feature is a general assistant, I do not pick a client at all until the product is smaller. The framework will not save a shapeless product. The thin client will not either. They will only make the shapelessness faster to deploy.

What I will not do

I will not run both in one service “so we can compare.” That comparison is a week you could spend on evals. I will not wrap Spring AI in our own interface on day one unless we already have two implementations. YAGNI applies to AI clients too. I will not choose Spring AI so the architecture review can say we are standard, then skip the kill switch because the bean looked production-ready. Ready-looking is not ready.

I will not choose a thin client so I can feel senior, then skip metrics because it is only 120 lines. Those 120 lines still hang the thread pool if I forget the timeout. Senior is the timeout, not the absence of a framework.

How I staff the choice

If the only person who understands Spring AI is the person who added it, that is a point against the framework in a small team. If the only person who understands the thin client is that same person, the point is a wash. I ask who gets the page at 3 a.m. and which file they can read while tired. In our Boot estate that file is usually a ChatClient config. In the gateway estate it is the gateway’s own docs. I will not invent a third file for taste.

Contractors change the math. A contractor who will leave in eight weeks should not introduce a framework the staff will not own. They also should not leave a clever client with no tests. I make them write the runbook paragraph before merge. That paragraph is the real design doc. If they cannot write it, we do not need their abstraction, Spring’s or theirs.

I also ask whether we are hiring Java people this year. If yes, Spring AI is a skill they can search. If we are not hiring and the team is three people who already have a RestClient they like, I will not make them learn a chat DSL for one button. Hiring plans are part of architecture. We pretend they are not, and then we are surprised when the bean is a museum.

The sentence I put in the review

Spring AI is worth it when we are a Spring team, we will have more than one model call, and we want the platform to own the client the way it owns JDBC. Calling the model ourselves is worth it when the path is already a gateway, the use case is one, or the protocol is a bad fit. In both cases the hard work is the same: flag, budget, tenant, evals. The framework does not remove that work. It only decides where the HTTP lives. I pick the place that matches the estate I already have, not the estate I wish I were building in a talk. If the review cannot name the estate, we are not ready to pick a client. We are ready to write the flag and the ten cases. I will happily delay the dependency for that. I will not delay those for a dependency.