Help me design…

“A lighthouse besieged by a sea serpent.”

Physically feasible Real LEGO® parts · No intersecting or floating parts · One connected component

Preprint 2026

BrickWright

Agentic Creation of Physically Feasible Brick Builds

Ava Pun ¹ ▪ Kevin Dai ¹ ▪ Sally Chen ¹ ▪ Maxwell Jones ¹ ▪ Ruixuan Liu ¹ ▪ Kangle Deng ² ▪ Changliu Liu ¹ ▪ Maneesh Agrawala ³ ▪ Deva Ramanan ¹ ▪ Jun‑Yan Zhu ¹

¹ Carnegie Mellon University ▪ ² Independent researcher ▪ ³ Stanford University

BrickWright enables pre-trained coding agents to generate high-fidelity, physically feasible LEGO® builds from text or image prompts. Its key idea:

Give agents a programmatic toolkit for creating and validating brick builds.

BrickWright logo BrickWright guides agents to generate brick builds by writing code with BrickLayer logo BrickLayer, a Python toolkit for brick build creation and validation. Agents guided by BrickWright produce higher-quality builds with fewer physical violations than agents with general tools. Additionally, they outperform specialized models on the text-to-brick task. BrickWright is especially useful for lightweight agents: on a sample of 512 prompts, it improves GPT-6 Luna's performance from 7% to 100% physically valid builds while boosting prompt adherence, at 2% the price of frontier models. BrickWright works with any agent backend; code is available on GitHub.

Better builds with BrickWright

Show intersections

Highlight colliding bricks

Prompt: “This is a white and black LEGO spaceship with a blue tinted canopy, featuring a blue-suited minifigure pilot, rear thrusters, and side-mounted weapons.”

GPT-6 Astra

Codex

BrickWright

GPT-6.1 Sol

Codex

BrickWright

GPT-6 Luna

Codex

BrickWright

Claude Haiku 5.5

Claude Code

BrickWright

Given the same prompt, BrickWright-equipped agents produce higher-quality builds with fewer physical violations than agents using general tools. The improvement is especially striking for lightweight agents Luna and Haiku. All BrickWright builds use real LEGO® parts, with no colliding (intersecting) or floating parts and one connected component. Red lines indicate brick intersections. Drag a model to rotate it.

Motivation

Frontier coding agents like GPT-6 Astra are becoming increasingly adept at modeling 3D objects, including brick builds. Agentic methods produce higher-fidelity builds and generalize better than specialized text-to-brick methods, such as BrickGPT and BrickNet. However, agents still face challenges in creating physically feasible builds. Here we define a build to be physically feasible if it uses only real parts, has no colliding (intersecting) or floating parts, and forms a connected structure if possible.

Existing methods

Show intersections

Highlight colliding bricks

Prompt: “This is a blue and gray LEGO flatbed truck featuring a detailed cab, four wheels with fenders, and a large gray cylindrical object mounted on its black studded cargo bed.”

Specialized text-to-brick models

BrickGPT

BrickNet

Agentic methods

Codex (GPT-6 Astra)

Codex (GPT-6.1 Sol)

Agentic methods better capture the request, but still produce physically infeasible builds. Both Codex outputs contain colliding bricks. Red lines indicate brick intersections. Drag a model to rotate it.

Method

To help agents produce physically feasible builds, BrickWright equips them with a programmatic toolkit for brick build creation and validation. Following the paradigm of structure-as-code, it guides an agent to write Python code that produces a brick build using a custom package called BrickLayer. BrickLayer provides a unified programmatic representation for LEGO® build files, enabling them to be easily understood and edited by humans and agents. Key BrickLayer API components include the following:

Discover parts

Search the part catalog by keyword. Keywords may target specified metadata fields, such as category and dimensions. You may also filter by specifying a field’s exact value.

search_parts(keywords, fields, filters)

Assemble geometry

Compose parts and sub-builds. Components may be placed with an exact pose or by automatically aligning typed connectors.

b = LegoBuild()
p = Element(ldraw_id, color)
b.add(p, position, rotation, relative_to, name, …)
b.connect(p1, p1.find_connector(…), p2.find_connector(…))

Check physical feasibility

Find collisions, floating parts, and connected components.

b.colliding_placements()
b.floating_placements()
b.connected_components()

To create a brick build from a text or image prompt, BrickWright provides a pre-trained agent with a set of tools—validate, render, and finalize—and prompts the agent to create a build with BrickLayer by writing and revising a single Python file in a plan-execute-validate loop. The validate tool provides feedback on the build’s physical feasibility, and the render tool allows the agent to inspect the build visually. When all feasibility tests pass and the agent is satisfied with the render, it calls finalize, completing the process.

BrickWright build creation loop

System (paraphrased)

Write a Python file, build.py, that uses BrickLayer to create a LEGO® build. This file should contain two functions: create_build(), which returns a LegoBuild, and run_tests(), which runs a set of tests against the build.
You are given the tools explore, validate, render, and finalize, alongside build-file editing tools. Calling validate runs both your custom tests and a fixed set of feasibility checks. Edit build.py until all tests pass and you are happy with the render, then call finalize. You may run explore at any time to investigate the BrickLayer API.
Here is the complete BrickLayer and tool documentation: […]
User

Build a red house with a blue roof and a brown front door, using 20 parts.

Step 1/11 Build request

Scroll to view diagram →

Input

User ▪ Text and/or image prompt System ▪ BrickLayer documentation; create-execute-validate instructions
Current step

Write build.py

import bricklayer
def create_build(): …
def run_tests(): …

validate()

Compiles, no collisions or floaters, expected number of components, passes custom tests?
render()

Multi-view or custom view
finalize()

Completed brick build
explore(snippet)

Optionally run a Python snippet to investigate BrickLayer

Inspect results & revise

Illustrative demo of brick build generation with BrickWright. This is a scripted conversation with real brick builds, renders, and validation results.

Experiments

We evaluate physical feasibility and prompt adherence on BrickNet-512, a dataset of 512 text prompts created by VLM captioning of human-made brick builds. As seen in the visual comparison below, BrickWright reduces colliding and floating artifacts for all agent backends and improves the coherence of builds created by lightweight agents.

Qualitative comparison

Show intersections

Highlight colliding bricks

Prompt: “This is a LEGO build of a blue, grey, and black all-terrain vehicle (ATV) or quad bike, featuring large treaded wheels, a roll cage, and translucent yellow headlights.”

Codex (GPT-6 Luna)

BrickWright (GPT-6 Luna)

Swipe the prompt or use the arrows to browse. Red lines show mesh intersections. Drag a build to rotate it.

1 / 8

Numeric results are given in the table below. Compared to vanilla agent baselines, brick builds produced by BrickWright-equipped agents exhibit greatly improved physical feasibility with broadly comparable token costs. This increase in physical validity is not accompanied by a significant decrease in complexity or fidelity; on average, BrickWright agents create builds with more bricks than vanilla agent baselines, while improving prompt adherence for lightweight models (Haiku, Luna) and maintaining it for powerful models (Sol, Astra). BrickWright-equipped agents additionally outperform specially trained text-to-brick models in overall validity, connectivity, and prompt adherence. The Articraft baseline, which is intended for digital 3D assets and does not handle brick build-specific constraints, yields comparable prompt adherence but worse physical validity at more than twice the cost of BrickWright.

Quantitative results

Agent	Method	Valid ↑	No collisions ↑	No floaters ↑	Components ↓	VQA ↑	Perception Encoder ↑	SigLIP 2 ↑	Parts	Unique parts	Checks ↓	Time ↓	Cost ↓
—	BrickGPT	0.990	1.000	0.990	2.67	0.0658	0.1547	0.0514	144.8	7.1	—	247s	—
—	BrickNet	0.318	0.320	0.943	1.03	0.6077	0.2846	0.6407	31.8	11.5	—	5s	—
Claude Haiku 5.5	Claude Code	0.328	0.428	0.387	3.59	0.6032	0.2963	0.7286	12.8	5.3	—	168s	$0.04
Claude Haiku 5.5	BrickWright w/o assembly API	0.822	0.889	0.828	2.07	0.5744	0.2940	0.7009	16.6	4.8	9.29	279s	$0.13
Claude Haiku 5.5	BrickWright w/o validator	0.586	0.744	0.775	2.99	0.6390	0.3012	0.7481	18.3	7.4	—	119s	$0.03
Claude Haiku 5.5	BrickWright	1.000	1.000	1.000	1.00	0.6523	0.3056	0.7660	21.5	7.6	2.89	143s	$0.05
GPT-6 Luna	Codex	0.086	0.131	0.135	10.11	0.6504	0.3025	0.7593	21.1	6.1	—	150s	$0.02
GPT-6 Luna	BrickWright w/o assembly API	0.996	1.000	0.996	1.04	0.3342	0.2586	0.4972	7.2	2.0	15.25	405s	$0.06
GPT-6 Luna	BrickWright w/o validator	0.529	0.561	0.840	2.03	0.6975	0.3111	0.7920	28.5	8.5	—	510s	$0.03
GPT-6 Luna	BrickWright	1.000	1.000	1.000	1.00	0.6945	0.3105	0.7766	27.4	7.8	4.62	415s	$0.03
GPT-6.1 Sol	Codex	0.641	0.654	0.781	2.00	0.7610	0.3241	0.8700	75.9	15.8	—	346s	$0.29
GPT-6.1 Sol	BrickWright w/o assembly API	1.000	1.000	1.000	1.01	0.7623	0.3228	0.8573	70.1	13.9	10.02	459s	$0.40
GPT-6.1 Sol	BrickWright w/o validator	0.779	0.787	0.990	1.03	0.7611	0.3240	0.8664	86.0	19.6	—	328s	$0.20
GPT-6.1 Sol	BrickWright	1.000	1.000	1.000	1.00	0.7610	0.3243	0.8669	75.8	16.6	2.48	251s	$0.19
GPT-6 Astra	Codex	0.693	0.709	0.822	1.87	0.7609	0.3237	0.8691	75.5	15.3	—	244s	$1.21
GPT-6 Astra	Articraft	0.684	0.693	0.844	1.77	0.7690	0.3249	0.8785	63.2	16.0	—	426s	$3.00
GPT-6 Astra	BrickWright	1.000	1.000	1.000	1.00	0.7695	0.3243	0.8698	89.3	18.9	2.40	267s	$1.28

We report the proportion of prompts yielding valid builds (no collisions or floaters), collision-free builds, and floater-free builds. Components, Parts, and Unique parts are the mean connected-component, part, and distinct part-ID counts. Checks is the mean number of validate calls. ↑ means higher is better; ↓ means lower is better. Bold and underline mark the best and second-best results within each agent. Time and cost are averages per prompt; costs use October 2026 token pricing, including cache savings.

Acknowledgments

This work is partly supported by the Packard Foundation, Sloan Foundation, and IITP grant funded by the Korean Government (MSIT) (No. RS-2024-00457882, National AI Research Lab Project). Ava Pun is supported by the Meshy Fellowship, Tata Consultancy Services Presidential Fellowship, and NSERC CGRS D. Maxwell Jones is supported by the Rales Fellowship.

Citation

If you’ve used BrickWright in your work, please cite us!

@misc{pun2026brickwright,
  title = {BrickWright: Agentic Creation of Physically Feasible Brick Builds},
  author = {Pun, Ava and Dai, Kevin and Chen, Sally and Jones, Maxwell and Liu, Ruixuan and Deng, Kangle and Liu, Changliu and Agrawala, Maneesh and Ramanan, Deva and Zhu, Jun-Yan},
  year = {2026},
  archivePrefix = {arXiv},
}