--- title: "Computer Use vs a BIM API: how AI agents drive CAD" description: "Can AI agents control CAD? Compare Computer Use on a GUI, a tool API or MCP on desktop BIM software, and a building program run and checked by a BIM kernel." url: https://buildify.cz/en/computer-use-vs-bim-api source: https://buildify.cz/computer-use-vs-bim-api.en.md language: en publisher: Buildify (Buildify Digital s.r.o.) dateModified: 2026-09-27 keywords: ["computer use CAD", "can AI agents control CAD", "AI agent BIM API", "MCP server BIM", "LLM tool calling BIM", "programmable BIM model", "BIM kernel checker", "GUI automation vs API"] license: "Volně citovatelné s uvedením zdroje / freely quotable with attribution" --- # Computer Use vs a BIM API: three ways an AI agent can build a model > An AI agent can click through a CAD interface, call the tool API of desktop BIM software, or write the building as a program that a BIM kernel executes and checks. All three can produce a model. They differ in what the agent can see, what a step costs, how errors come back and whether the result can be repeated. - In BIM Harness the agent writes a script in a building dialect of more than a hundred verbs - A family house builds on the kernel in about 7 seconds - the language model is the slow part - Errors come back as named findings such as door.obstructed, not as a screenshot to interpret - The same script against the same kernel builds the same model Get access to BIM Harness: https://buildify.cz/en/bim-harness#waitlist ## Can AI agents control CAD? > Computer Use automates interaction with software. BIM Harness makes the building model itself programmable: the agent writes a building program, and a BIM kernel executes it and returns named findings. Yes, AI agents can control CAD - and there is more than one way to do it. A multimodal model can look at the screen and operate the mouse and keyboard the way a person would. A model can call functions that desktop BIM software exposes through a plugin or a Model Context Protocol (MCP) server. Or a model can write the building directly as code in a domain language, which a BIM kernel turns into a model. These are not competing brands of the same idea. They sit at different levels. The first automates the user interface. The second automates the software behind the interface, one call at a time. The third moves the work to where the model is actually defined: the object model and its rules. Which one fits depends on what the software exposes and what you need to get back. ## Three routes, compared on the things that matter for a model The comparison is about the route, not about any one product. Each route can be done well or badly; the table shows what each one structurally gives the agent to work with. | Aspect | Computer Use on a GUI | Tool API / MCP of desktop BIM | Building program + BIM kernel | | --- | --- | --- | --- | | What the agent produces | Clicks, keystrokes and dialog inputs | A sequence of tool calls | One script in a building dialect | | What the agent sees | Screenshots of the interface | Return values of each call | Named findings from a checker, split into author and engine findings | | Cost per step | A model call per screen interaction | A model call per tool call | The kernel runs the whole script; the model is called to write and repair it | | Error recovery | Notice a misclick or an unexpected dialog, then retry | Read an error message from the call | Read a named finding (door.obstructed, clash.hard) and make a targeted edit | | Repeatability | Depends on screen state, window layout and timing | Depends on call order and application state | The same script against the same kernel builds the same model | | Versioning | A recording of actions | A log of calls | A text file you can diff, read and edit | | Where it fits best | Software that exposes its functions only through a GUI | Driving an existing desktop BIM installation | Generating and regenerating a whole building with its systems | No cost multipliers are quoted here on purpose. What is certain is the structure: a GUI route pays for every interaction, a program route pays for writing the program. ## Computer Use: valuable where the GUI is the only door Computer Use lets a model operate software the way a person does: take a screenshot, decide where to click, open a menu, fill in a dialog, confirm, take another screenshot. For a great deal of software this is the only way in. Legacy tools, internal applications and programs without any scripting interface expose their functionality only through the screen. For them, an agent that can see and click is a genuine step forward. For building models, the limits come from the interface rather than the agent. A wall placed through a dialog is a wall placed through a dialog - the agent learns what happened by looking at pixels again. Each step is a model call. An unexpected pop-up, a different window size or a slow redraw changes what the next screenshot shows. And the record of the work is a series of interactions, not a description of the building that someone can read and change. ## A tool API or MCP on existing desktop BIM software The second route skips the pixels. The desktop application, or a plugin for it, exposes functions such as "create wall" or "list rooms", and the language model calls them as tools. MCP servers for established desktop BIM tools - Autodesk now ships one - are the best-known example of this approach. It is a sensible way to add an agent to a workflow that already lives in that software. The agent now gets structured return values instead of screenshots, which is a real improvement. The shape of the work stays the same, though: the model issues calls one by one against a running application, and the model of the building exists only as the state that those calls leave behind. The agent works inside a tool that was designed around a human operator. ## A building program, executed by a kernel with a checker > In BIM Harness the AI writes a program, not geometry: a script in the Harness building dialect, with calls such as api.wall, api.space and api.door, which the Harness parametric kernel executes and checks. The third route treats the building as something you write rather than something you click. The agent does not operate an application. It writes a script, the kernel builds it into a parametric object model, and a checker reports what is wrong in terms the agent can act on. The kernel is the fast, cheap part. In a real production run of a family house, building the model in the browser took about 7 seconds and laying the MEP about 6 seconds; nearly all of the wall-clock time was the language model thinking. This only works because the kernel was written for it. BIM Harness stands on its own parametric kernel, written from scratch, not on a plugin for someone else’s CAD. The same Scripting API that the agent builds with is available to people in the editor: whatever the agent can do, so can you. ## What the agent writes A shortened part of a real script, tested against the kernel. The door is attached to a wall by its id - a relation, not a pair of coordinates the model has to get right. ```js const ground = api.ensureStorey('Ground floor', 0); const W = 12, D = 8, H = 3.4; const outline = [[0,0],[W,0],[W,D],[0,D]]; api.floorAssembly({ slabOutline: outline, wallOutline: outline, storey: ground }); const south = api.wall({ start: [0,0], end: [W,0], storey: ground, height: H }); api.wall({ start: [W,0], end: [W,D], storey: ground, height: H }); api.wall({ start: [W,D], end: [0,D], storey: ground, height: H }); api.wall({ start: [0,D], end: [0,0], storey: ground, height: H }); const partition = api.wall({ start: [W/2,0], end: [W/2,D], storey: ground, height: H, thickness: 0.15 }); api.space({ outline: [[0,0],[W/2,0],[W/2,D],[0,D]], storey: ground, height: H, name: 'Hall', type: 'hall' }); api.space({ outline: [[W/2,0],[W,0],[W,D],[W/2,D]], storey: ground, height: H, name: 'Study', type: 'office' }); api.door({ wall: partition.id, offset: D/2, width: 0.9 }); api.door({ wall: south.id, offset: W/4, width: 1.4 }); api.window({ wall: south.id, offset: 3*W/4, width: 1.2, sill: 0.9 }); ``` Fourteen lines describe a storey, its walls, two rooms as IFC spaces, two doors and a window. Through a GUI, the same result is a long series of separate interactions. ## Where the difference shows up in practice - **Observability** - A screenshot tells the agent what the screen looks like. A checker tells it what is wrong with the building: clash.hard, door.obstructed, stair.throughFabric, room.accessEnvelope. Findings are machine-readable and split into author findings the model should fix and engine findings. - **Repeatability** - The language model is not deterministic, but execution is. The same script against the same kernel builds the same model, and MEP layout and clash resolution are done by deterministic resolvers, not by the language model. - **Cost per step** - In a GUI route every interaction is a model call. A program is one artefact that the kernel runs in seconds; the model is paid to write and repair the program, not to push each wall into place. - **Error recovery** - When a verb cannot do what it was asked, the kernel refuses by name. In a production run the kernel answered "api.gable: the roof given does not cover this wall ... It was not built." The model rewrote that part of the script. - **Versioning** - A script is text. It can be diffed, reviewed and edited. The agent reads its own script back and changes it with targeted edits instead of rewriting everything or replaying a session of clicks. ## How an error is fixed on the program route The loop the agent runs is short and explicit. Nothing is committed until the trial passes. 1. **Trial build** - The kernel builds and checks the script. Nothing is committed to the project. 2. **Named findings** - The checker returns findings such as door.obstructed or clash.hard, each tied to what caused it. 3. **Repair plan** - Findings feed a repair plan for the next round, so the agent knows what to change and why. 4. **Targeted edit** - The agent reads its script back and edits the lines concerned, leaving the rest of the building alone. 5. **Live build** - The corrected script runs as the live build and becomes the model you open, edit and export to IFC4X3. ## The agent next to the model it built ![BIM Harness editor with the AI assistant panel beside a generated multi-storey building](https://buildify.cz/screenshots/harness-ai.webp) The AI assistant panel next to a generated building in the BIM Harness editor. The interface is in Czech; the model behind it is an IFC model you can open, edit by hand and export. > Computer Use automates interaction with software. BIM Harness makes the building model itself programmable. ## Which route to choose If the software you need to automate has no API and no scripting interface, Computer Use is often the only practical route, and it is a good one. If your team already works in an established desktop BIM tool and wants an assistant inside it, a tool API or MCP server on that tool keeps everyone in familiar software. If the goal is to generate a whole building - storeys, structure, rooms, façade, roof and MEP systems - and then change it again and again, the program route is the one that scales. The model of the building lives as a readable script and a parametric object model, the kernel does the geometry, and the output is open IFC4X3 that established desktop BIM tools and CDEs pick up. ## Frequently asked questions ### What is the difference between Computer Use and an API for CAD? Computer Use lets an AI model operate CAD software through its graphical interface: it takes screenshots, moves the mouse, opens menus and fills in dialogs, with a model call for each interaction. An API route lets the model call the software’s functions directly and read structured results. BIM Harness goes one level further: the model writes a building program that a BIM kernel executes and checks as a whole. ### Can AI agents control CAD and BIM software? Yes. Agents can drive CAD through the screen with Computer Use, call functions exposed by desktop BIM software through a plugin or an MCP server, or write the building as code for a BIM kernel. BIM Harness uses the third approach: the agent writes a script in a building dialect of more than a hundred verbs, and its own parametric kernel builds and checks the model. ### Is Computer Use a bad way to automate BIM? No. Computer Use is valuable for software that exposes its functionality only through a graphical interface, where no API exists. Its limits for building models are structural: the agent sees screenshots rather than the model, pays a model call per interaction, and depends on screen state. Where a programmable model and a checker exist, the agent can work with structured findings instead. ### How is BIM Harness different from an MCP server for desktop BIM software? An MCP server is an example of the tool-API approach: a language model calls functions of an established desktop BIM application one at a time. BIM Harness does not drive another application. Its agent writes a building program that the Harness parametric kernel, written from scratch, executes and checks. The results meet at IFC: Harness exports IFC4X3 that Autodesk and Archicad tools pick up. ### Why is a building program more repeatable than GUI automation? A GUI session depends on screen state, window layout, dialogs and timing, so replaying it can go differently. In BIM Harness execution is deterministic: the same script against the same kernel builds the same model, and MEP layout and clash resolution are done by deterministic resolvers. The language model that writes the script is not deterministic, but what it wrote runs the same way every time. ### How does the agent find out that something went wrong? The BIM Harness checker returns named, machine-readable findings, split into author findings (design mistakes the model should fix) and engine findings. Examples include clash.hard, door.obstructed, stair.throughFabric and room.accessEnvelope. When a verb cannot do what it was asked, the kernel refuses by name and does not build it. Findings feed a repair plan, and the agent edits the lines concerned. ## Related - [what AI-native BIM means](https://buildify.cz/en/ai-native-bim) - [scripting a BIM model with the same API the agent uses](https://buildify.cz/en/bim-scripting) - [how the BIM agent loop works step by step](https://buildify.cz/en/ai-bim-agents) - [text-to-BIM from a written brief](https://buildify.cz/en/text-to-bim) - [why a generated 3D mesh is not a BIM model](https://buildify.cz/en/text-to-3d-vs-text-to-bim) ## Let the agent write the building, not click it BIM Harness gives the AI a building dialect, a parametric kernel and a checker - and gives you the same Scripting API, the full browser editor and open IFC4X3 at the end. Get access to BIM Harness: https://buildify.cz/en/bim-harness#waitlist --- Zdroj / Source: https://buildify.cz/computer-use-vs-bim-api · Kontakt: kluch@buildify.cz · Buildify Digital s.r.o., IČO 29541743, Korunní 2569/108, 101 00 Praha 10, Česká republika.