Pointing Beats Describing
How Quey captures what you see and hands it to your AI agent without losing anything in translation — a deep look at selectors, shadow DOM, and structured context.
When you tell an AI agent to “fix the button hover,” you've already lost something. The agent has to guess which button, infer which hover state, find the element in a codebase it can only see as text, then make a change without knowing what the original looks like. Each step is a guess that compounds the previous one.
This is the translation problem. Visual information — the exact shade, the precise timing, the specific element in a sea of similar elements — collapses when it passes through natural language. Screenshots help, but only marginally. The agent still has to infer what you're pointing at.
Quey was built around a single conviction: pointing beats describing. Here's how we make that work.
The translation problem
Precision evaporates in translation. “The button has a blue background” is a description. { background: '#527fd9', cssVariable: '--palette-brand-blue' } is a fact. An agent can grep for the CSS variable. It can update the design token. It can trace the value back to wherever it's defined. Descriptions invite interpretation; structured data eliminates it.
The harder something is to describe — an animation timing curve, a stacking context issue, a subtle spacing inconsistency — the more valuable it is to just point at it. The harder it is to describe precisely, the more you lose in the description.
What capturing really means
When you click an element with Quey, we don't take a screenshot and call it done. We capture a structured snapshot of everything an agent would need to understand that element in context: the DOM selector, the semantic component path, the computed CSS values, the design tokens with their CSS variable names, the source file location, and the element's role in the layout.
The output is a CapturedElement — a versioned typed object that has evolved across four schema revisions. Not a prose description of these things. The actual values, in machine-readable form, ready for an agent to act on without guessing.
element (tag, role, accessible name, bounds), identity (component name, state tags, file path, line), location (selector path, semantic path, class list), design (colors, font, radius, layout), and styleEditor (editable properties and their current values).The selector problem
Generating a stable CSS selector for an element on a live page is genuinely hard. DOM paths shift constantly as React re-renders. IDs are often absent, auto-generated, or collision-prone. In a Tailwind codebase, class names are utilities, not identifiers —.flex.items-center.gap-2 describes half the page. A path like div:nth-child(3) > div:nth-child(2) > button breaks the moment a component re-orders its children.
We walk the DOM upward from the selected element, scoring each candidate anchor by stability and specificity. A data-testid attribute beats a semantic tag. A unique class combination beats a numeric position. The selector we emit is the shortest path that uniquely identifies the element in the current document and is most likely to survive an agent's edits.
Alongside the CSS selector, we emit a semanticPath: a human-readable component chain derived from the React fiber tree. ProductCard > ActionRow > PrimaryButton tells a story that.flex.gap-4 > button.rounded-md doesn't. The CSS selector gives the agent something to grep; the semantic path gives it something to understand.
Reading the React fiber tree
React component names don't live in the DOM. At runtime, a <PrimaryButton> is just a <button> — the component name is only accessible through React's internal fiber tree. In development mode, the browser exposes__REACT_FIBER_REFS__, a registry that maps DOM nodes back to their fiber instances.
We walk up from the selected element's fiber, collecting the nearest named components and their source locations. The result is a component stack with file paths and line numbers — the exact coordinates an agent needs to open the right file and start editing.
Why we chose the shadow DOM
The Quey toolbar renders inside a shadow root attached to <html>, not <body>. Any UI that renders directly into the document inherits the page's CSS. On a site with a global box-sizing: content-box override, your carefully measured toolbar becomes the wrong size. On a site with a font-size reset on *, your type scales wrong. On a site with aggressive z-index stacking, your panels disappear behind content they should float above.
The shadow root gives us a completely isolated style scope. Nothing from the page leaks in; nothing from the toolbar leaks out. The extension renders on any site, in any CSS state, without needing to know anything about the host.
There's one deliberate exception: the selection overlay. The blue selection box that traces element boundaries renders in a separate <div> outside the shadow root, using imperative DOM. This is because backdrop-filter doesn't compose across shadow boundaries the way we need, and precise viewport-relative positioning is simpler when you're not fighting shadow DOM stacking. The overlay is a single self-terminating requestAnimationFrame loop — no React, no state, just direct DOM writes at paint time.
We also attach both roots to <html> rather than <body>. Single-page applications frequently replace the entire <body> on route changes. Attaching to <html> means Quey survives navigations that would otherwise unmount and destroy it.
What we actually send
When you click Copy in Quey, the clipboard receives three simultaneous formats:
application/x-quey+json
The full CopiedElementPayload — a versioned object (currently schema v4) containing the complete CapturedElement, any pending style edits in thedraftState, your typed instruction in agentContext, and a structured diff of what changed vs. what was there before. This is the machine-readable brief: an agent that can parse this format gets richer context than any text description could provide.
text/plain
A hand-crafted prose prompt: selector, component path, file location, computed values, and your instruction, formatted for direct paste into Claude, Codex, or any chat interface. Most agents today read from a chat input, not directly from clipboard data — this format covers that gap.
image/png (region captures only)
In region-capture mode, the cropped screenshot of the selected area, composited from achrome.tabs.captureVisibleTab call made by the service worker. The content script can't call this API directly — that's one of the few things that requires the background context.
// The versioned clipboard payload — schema v4interface CopiedElementPayload { schemaVersion: 4; kind: "toolbar-style-change"; capture: CapturedElement; draftState: Partial<Record<StylePropertyName, string>>; target: { selector: string; semanticPath: string[] }; prompt: string; // the plain-text agent handoff agentContext?: { instruction: string };}The MCP bridge
The clipboard is synchronous and human-mediated. For anything that requires back-and-forth iteration — a style system audit, a component refactor, a multi-step visual fix — the copy-paste loop becomes friction.
When quey-mcp is running locally on port 4747, every element capture you make in the browser is also posted to the bridge automatically. A Claude Code or Codex session connected to the MCP server can pull the latest capture at any point, without you doing anything — no copy, no paste, no context switch.
This changes the dynamic. Instead of instructing an agent step by step, you click through your UI, build up a set of captures, and let the agent process them in batch. You become the pointer; the agent becomes the writer. The handoff is no longer a moment — it's a continuous stream.
QUEY_AUTO_RUN=1 and QUEY_CODEX_CWD=/path/to/repo and the MCP runner will kick off an agent automatically when a new capture arrives. Point at something, type your instruction in the Agent panel, and the edit starts without you opening a second window.Style editing as a capture mode
Quey's style editor isn't a standalone design tool — it's a capture mode. When you open the Style tab, select an element, and adjust its background color or font size, those changes are tracked as a styleDraft. The draft is applied live to the element's inline style so you can see the effect in real time — but nothing is committed until you choose to copy or apply.
When you copy from the Style panel, buildEffectiveCapture() merges the draft into the captured snapshot before writing to the clipboard. The payload reflects the world as you're proposing it, not as it currently is. The agent receives a before-and-after diff — what the element looks like now and what you want it to look like — so it can write a targeted change instead of rebuilding from scratch.
The capture schema has a version number for a reason
The CopiedElementPayload is currently at schema version 4. Each version added a new layer of context:
v1 — Selector and tag name only.
v2 — Computed styles: background, foreground, font, radius.
v3 — Source hints: file path, line number, component name.
v4 — Agent streaming: MCP bridge, structured diffs, agentContext.
The version number exists because payloads move between tools. The extension writes them; agents, scripts, and future tooling read them. Schema versioning means a v3 payload read by a v4 parser degrades gracefully instead of breaking silently.
Version 5 is being designed now. The planned addition is region context: the ability to capture a section of the page as a whole — its layout relationship, the elements it contains, and how they relate to each other — rather than a single element in isolation. One click, one complete brief for an agent redesigning a component tree rather than tweaking a single node.