Ask, or paste an OpenSTA timing report
Try
How it works
- Plan. The LLM picks tools as JSON. Each step is checked against the tool's input schema; a bad plan falls back to a rule router.
- Tools.
search_docs(BM25 + MiniLM embeddings, RRF fusion, registry routing),lookup_command(254 commands and their documented flags),parse_timing(OpenSTAreport_checks). - Answer. Cited, from the retrieved passages and tool output only.
- Verify. Every command, flag, flag value and decimal number must appear in a source or in the registry (481 commands parsed from OpenROAD and OpenSTA
define_cmd_args). Sentence support is lexical overlap, so contradictions are flagged for a human. One rewrite if a check fails. - Real runs. A Python CLI runs the same loop against OpenROAD on a real checkpoint, executes allowlisted steps and accepts a step only if timing and area assertions pass. See the Real OpenROAD runs tab.
Answer
Trace
- No run yet.
Sources
The agent on a real design
Design: gcd on Nangate45 from OpenROAD-flow-scripts, 0.25 ns clock, post-CTS checkpoint. Case 2 adds a 0.1 ns hold uncertainty to the same checkpoint (a common signoff margin). Each model proposes one step at a time. The verifier blocks anything off the allowlist or not in the registry. Steps that pass run in OpenROAD (Docker, no network), then timing and area are re-measured. A step is accepted only if every assertion passes. Up to 4 attempts.
Local model: Qwen2.5 7B (Q4) through Ollama on a laptop CPU, with no data leaving the machine. Hosted model: Llama 3.3 70B on Cloudflare Workers AI. Rules: a fixed sequence, the no-LLM baseline.
Retrieval evals
Does the right module doc come back? Query vectors are precomputed, so this runs in the browser in under a second.
Retrieval or long context: cost per question
Three ways to give the model the docs: the top 5 chunks, the whole top module (an agent reading one file), or the whole corpus in one prompt. Token counts use about 4 characters per token.
Guardrail eval: invented commands, flags and values
Fourteen answers I wrote, eight with a planted error: a flag the command does not take, a command that does not exist, or a flag missing its value. The verifier must flag the eight and pass the rest. Planted errors are easier than real ones, so treat this as a regression test.
Tool eval: timing parser
Hand-written fixture, plus the real gcd post-route report checked against OpenROAD's own report_worst_slack.
Knowledge base
Command registry
Parsed from the tcl blocks and option tables in each module doc. The verifier treats this as the list of real commands and flags.
| Command | Module | Documented flags |
|---|