Model proxy

Everything in a run can read the run’s environment: your repository’s setup steps, the commands the agent runs, and the agent itself, if something it read talked it into it. So Ryph keeps your model keys out of runs. Every model call an agent makes goes through Ryph’s model proxy.

Your provider keys stay sealed in Ryph. A run gets a key that works only for itself, and only for the models its agent may use. The proxy checks each call, sends it on with the real key and counts what it cost, against the run, the agent and the task. When the run ends, its key stops working.

What a run holds

• A key for itself. Ryph makes one when the run starts, and puts it where the provider’s tools look for it, such as ANTHROPIC_API_KEY, with the base URL pointed at the proxy. The agent calls the model as usual and doesn’t notice the difference.
• Nothing that outlives it. The key stops working when the run ends, and after 12 hours at most, which RYPH_VIRTUAL_KEY_TTL_MS changes.
• No provider key. Someone who steals the run’s key gets a key for one run and its models, and only until the run ends.

What the proxy checks

• The model. Every call that buys tokens has to name a model the agent may use. A call for any other model, or one that names no model, is refused.
• The endpoint. For Anthropic and OpenAI, a run can only send messages, count tokens and list models. Every other part of those APIs is refused, and the proxy drops headers that could reach past the call, such as an OpenAI organization ID.
• The spend. A run at its spend cap, or a deployment at its ceiling, gets its calls refused. More on that below.
• Where it sends calls. The proxy doesn’t follow redirects, so a provider’s answer can’t send your key somewhere else.

Add a provider key

Go to Model keys and click Add key. Adding and changing keys takes an operator or an owner.

1. Pick the provider

   Anthropic and OpenAI are built in. Any other provider has to be added first, as a custom provider.

2. Paste the key

   Ryph seals it, and doesn’t show it again. The list shows its last four characters, so you can tell keys apart.

3. Give it to agents

   Open Agents, pick an agent, click Edit and go to Model. Pick the Provider key, tick the Allowed models, and choose the Default model, which a task gets when it doesn’t pick one.

Archive is the off switch for a key: runs that use it lose it on their next call. Replace secret puts a new key in place of the old one, and running runs use it from their next call.

Custom providers

Under Custom providers, add any endpoint that speaks OpenAI’s API: Groq, Together, Fireworks, OpenRouter, DeepSeek, Azure OpenAI, Google AI Studio, Cloudflare Workers AI or your own vLLM. Name the models it serves, their context windows and, if you want costs to add up, their prices. The key can go in a bearer token or a header of the provider’s choosing.

The endpoint has to be a public HTTPS address. For a model server on your own network, set RYPH_ALLOW_PRIVATE_MODEL_ENDPOINTS=1.

A run can call any path under a custom endpoint’s address, with your key attached. The proxy can hold Anthropic and OpenAI to the endpoints agents need because it knows their APIs, and it can’t do that for a provider it doesn’t know. Add providers you’d trust with the key.

Your ChatGPT plan

When Ryph runs on your machine, it can spend the usage your ChatGPT plan includes. Click Use your ChatGPT plan, then Continue with ChatGPT, and sign in. Runs on the plan count as included, and spend caps don’t apply to them. When the plan’s limit is reached, its calls fail until it resets, and Ryph doesn’t switch to another key.

Know what it costs

Ryph records every call: the tokens in and out, cache reads and writes, reasoning tokens and how long it took, along with its run, agent and task.

• The run page shows the run’s calls, tokens and cost for each model.
• Usage, for operators and owners, shows the whole deployment over a window, 30 days by default: totals, and the split by model, agent, provider key and day, with the tasks that cost the most.

Costs are at list price, from Ryph’s price list, which is worked out when you look. So a corrected price corrects past costs too. A custom model with no price is shown as unpriced, and isn’t counted as free. Your bill is the provider’s.

Cap what agents spend

For each agent, open Agents, pick the agent, click Edit and go to Limits. Spend sets what one run may spend on model calls, in US dollars at list price. At the cap, Ryph stops the run. Model calls in flight limits how many calls a run can have open at once, and Wall clock how long it can take. A deployment can set the highest caps an agent may have, and changing an agent’s limits takes an operator or an owner.

For the deployment, RYPH_MODEL_SPEND_CEILING_USD sets a ceiling on what all model calls together may spend over the last 30 days, or over RYPH_MODEL_SPEND_WINDOW_DAYS. At the ceiling, every call that would spend is refused, and no new task can start. It’s set in the environment, so nobody can raise it from the web app.

Ryph’s own model calls

Some of Ryph’s own work uses a model: checking memories against each other, and turning documents into memories. An administrator picks the key and model for it under Platform, in Background model. That work gets keys from the proxy too, is held to the models you picked, and shows up in Usage on its own line. Without a background model, Ryph skips the parts of that work that need one.

What to keep in mind

• The provider sees what the agent sends. That includes your code and anything the agent read. The proxy decides which models a run can use, and the prompt goes to that model’s provider as it is.
• Costs are estimates. They’re at list price, and don’t know about your discounts or credits.
• Only the proxy holds the keys. In Networked Compose and on Kubernetes, the proxy is its own service, and the key that seals provider keys is its alone. The control plane can’t open them.