Coding with AI when cloud tools are not allowed

How good are open coding models?

According to measurements by Epoch AI, the best open models are about four months behind the leading closed models. Many are released under free licences such as Apache 2.0 or MIT, some under their own licences with conditions for very large providers; check the licence file before use. Be careful with rankings: the best-known test, SWE-bench Verified, is considered exhausted, and vendors’ own figures are often well above independent measurements. A test with tasks from your own code is what you can rely on.

What fits on which hardware

GPUs	GPU memory	Coding model	Alongside
3× RTX PRO 6000	288 GB	up to about 80 bn parameters on two cards	third card for suggestions while typing and code search
2× H200	282 GB	up to about 80 bn with plenty of room for long sessions, just under 300 bn possible	–
2× H200 + 1× RTX PRO 6000	378 GB	as 2× H200	separate card for suggestions while typing and code search
8× H200	1,128 GB	the best open models	enough for several models

General estimate based on model cards, recipes of the inference software and published measurements, as of October 2026; not a commitment for any particular configuration. This refers to mixture-of-experts models in 8 bit.

How many developers can work at the same time?

That depends on the model and on how long the sessions get. A coding agent sends the entire history so far with every step, so above all the server has to read in a lot of text quickly. As a rough order of magnitude: on two H200 or three RTX PRO 6000, a model in the 80 bn class handles 15 to 40 agent requests at the same time. Since a team never works all at once, that is enough for a few dozen developers. Figures for questions in a chat cannot be transferred; agents put many times more load on the server. Only a measurement with your own tasks is reliable.

Which tools work without the cloud?

Most open tools can be switched to your own OpenAI-compatible interface. With some well-known products, parts still run through the vendor.

Tool	Without the cloud
Cline, Kilo Code (VS Code)	yes
Aider, OpenHands, Codex CLI	yes
Claude Code	yes, via the Anthropic interface of the inference software; switch off telemetry
Zed	agent yes; check suggestions while typing
GitHub Copilot	limited: chat and agent yes (since VS Code 1.122); suggestions while typing need GitHub
JetBrains AI Assistant	limited: own models yes; clarify operation without a connection to JetBrains
Cursor	no, requests run through Cursor’s servers
Continue, Roo Code	no longer maintained

What must the interface support?

Coding agents read files and run commands through tool calls. For this, the interface needs tool calling, streaming, long contexts and caching of repeated input; otherwise performance collapses. Suggestions while typing need a separate small model (1.5 to 7 bn parameters): it answers in fractions of a second and is even better at this than large models. Code search needs an additional embedding model. Configuring the inference server correctly for all this is a job for specialists.

How much does coding with AI consume?

Agents consume orders of magnitude more than a chat: one study measures around a thousand times more tokens for agentic tasks than for questions about code. For its own tool, Anthropic states an average of about 13 US dollars per developer per working day; that is tens of millions of tokens, most of them from the cache. On a server on your premises, what counts is therefore not the volume but how many requests run at the same time. Per-person quotas, as cloud licences have them, are enough for questions about code, not for agents.

What works less well than with cloud tools

• Long tasks: on tasks with many steps, the gap to the leading models is larger than on short ones.
• Speed: under load, a server on your premises often delivers 10 to 30 words per second per request; cloud services several times that.
• Very large projects: the agent searches the code instead of reading all of it. Long sessions cost memory and therefore concurrency.
• Setup: a wrongly configured format for tool calls is the most common source of errors.

Is the code safe on a server on your premises?

Safe from leaking to AI providers, yes. The server alone does not protect against manipulated instructions in code, documentation or dependencies: numerous vulnerabilities of this kind have been published for well-known coding tools. Agents therefore belong in an isolated environment, without automatic approval of commands, with limited network access and without access to credentials for production systems. The tools’ telemetry is a separate channel to the outside and should be switched off.

Who owns the code the AI writes?

In Germany, copyright only protects personal intellectual creations (§ 2 Abs. 2, § 69a Abs. 3 UrhG, the German Copyright Act). Purely machine-generated code is generally not protected; the more a human shapes it, the more likely protection becomes. One risk remains: a model may reproduce third-party code with licence obligations word for word. In the US, an appeals court dismissed copyright claims on this in September 2026; claims based on open-source licences are still pending. A licence scanner in the build pipeline helps. Incidentally, the model’s licence only governs the model, not the code it produces.

What does the EU AI Act require?

Coding tools are, as a rule, not high-risk systems. Art. 4 still applies: if you give developers AI tools, you must take measures that promote their AI literacy, for example on the limits of the models, manipulated instructions and reviewing generated code. AI literacy: training under Art. 4 →

How does this work with basebox?

On the server: alongside the AI model for the whole organisation, a coding model runs on your GPUs. We install the best, most capable models that suit your hardware. Your developers keep their tools and connect them via the OpenAI-compatible API of basebox, each person with their own access. We work out with you how many developers can work at the same time.

Not legal advice, no liability: This guide reflects the state of our research on the date given. Models, tools and licences change quickly. We accept no warranty and no liability for accuracy, completeness or timeliness.