New requests keep piling up: route traffic based on tenant IDs in JWTs instead of URLs, scrub sensitive info like Aadhaar and PAN numbers before writing logs, or set up a backup model when your main AI service runs slow.

None of these are new problems. Your team has solved each of them before. In another application. In another language. Probably more than once.

And now you will solve them again.

The logic nobody owns, but every app carries

Look inside most enterprise applications. Beyond the business logic, you will find a familiar set of extras.

This is traffic logic. It has little to do with what the application actually does. Yet it lives inside the code, written by the app team, tested with the app, and shipped with every release.

Now multiply that by every application in the estate. Java here. Python there. A Node.js service someone wrote three years ago. Each one carries its own version of the same rules. Each one implements them slightly differently.

The same logic gets rebuilt in every application. In every stack. Through every release.

We moved it to the proxy. The mess came along.

Experienced teams will say this is a solved problem. API gateways, ingress controllers, service mesh, and proxy scripting all promised to pull traffic logic out of the app. Many of them did.

But the logic did not get simpler. It just changed address.

Today it lives as snippets, scripts, and include files spread across dozens of proxy instances. Each one tuned by hand for a specific app or a specific incident. Ask who owns a particular rule, and the answer is often someone who has since left.

Moving logic to the proxy solved duplication. It did not solve governance.

A new gateway for every new problem

The other common response is to add another tool. An API gateway for APIs. A separate gateway for AI traffic. A DLP product for sensitive data. A translation layer for gRPC.

Each tool brings its own config model, its own console and its own team to run it. Often, one more hop in the request path too.

Meanwhile, the proxy layer that every request already passes through does far less than it could.

Every team feels it differently

The problem does not sit with one team. It shows up as a different frustration in each one.

Proxy and infrastructure admins. Every new requirement becomes a one-off config block or a custom script on a specific instance. Over time, no two instances look the same. Nobody wants to touch the production config, because nobody is fully sure what else depends on it.

SRE and operations teams. You need to shift traffic to a new endpoint. Simple enough. But the routing lives inside the app, so you raise a ticket, wait for the app team, and wait again for their release window. An operational decision moves at development speed.

Security teams. Sensitive data leaves through dozens of applications and APIs. Each handles it differently. There is no single point where you can say, with confidence, that a raw identifier never goes out.

Compliance and audit teams. An auditor asks how customer PII is protected. The honest answer is spread across many codebases, some written by third parties. Proving it is harder than doing it.

Application developers. You were hired to build features. Instead, a good part of the sprint goes into routing code, SDK upgrades, and masking hooks. Infrastructure decisions keep finding their way into application code.

AI is making it worse, and faster

Then came AI. And the same pattern repeated itself, at speed.

Every team that adds an AI feature makes its own choices. Which provider to call. Which SDK to use. Where to store the keys. How to handle a slow or failed response. How many tokens a tenant can burn in a day.

Each application answers these questions on its own. So the estate ends up with many ways of calling models, and no common view of cost, usage, or risk.

Switching providers becomes a code change. Adding a self-hosted model becomes a project. Setting a token budget means touching every app that calls a model.

Then there is the data question. What customer data is going into those prompts? An email ID here, a mobile number there, a PAN or account number pasted into a support query. Once it reaches an external provider, it is out of your hands.

Modernisation stalls when both sides must move together

Many enterprises run newer services on gRPC while older consumers still speak REST over HTTP/1.1. Or the reverse.

Bridging them usually means changing the client, the service, or both. In the same release window. Coordinated across teams that run on different calendars.

So modernisation waits. The legacy interface stays, because nobody can afford a rewrite on both sides at once.

The real cost: every change is a release

Duplicated code is the visible part. The deeper cost is what it does to change.

When traffic behaviour lives inside applications, changing that behaviour means changing the applications. A new routing rule, a new masking pattern, a new model provider. Each one turns into multiple code changes, multiple test cycles and multiple releases.

It also means masking happens late. By the time an application masks a value, the raw data has already crossed the network. It has already sat in a request buffer. It may already have reached an access log. The later the check, the larger the exposure.

Even the rules that already sit in the proxy carry risk. Each change is a hand edit, with no consistent way to validate it, roll it out in order or roll it back.

The result is an estate that is slow to change, hard to govern and difficult to prove.

The question worth asking

Almost every request in your estate already passes through a common layer. The proxy or gateway that sits in front of your applications sees every call before the app does.

Yet most of the logic that should live there still lives inside the apps.

So it is worth stepping back and asking a few simple questions.

If these questions feel uncomfortably familiar, you are not alone. Most enterprise teams are living with this today. They’ve stopped seeing it as a problem because it’s always been this way.

It does not have to stay that way.

We have been working on this exact problem. In an upcoming webinar, we will walk through how we approached it and where we landed.

Bring your questions. And if you have solved this differently, or know how you would want it solved, we would like to hear from you.