Google’s October 30, 2026 Gemini API release notes move the Computer Use tool to general availability across the Interactions and GenerateContent APIs. For developers implementing it, an important integration detail remains: a returned click position needs to be converted to the dimensions of the environment receiving that click.
Google’s October 30 Computer Use release note
Keep the model position and screen position separate
Google’s implementation guide describes coordinates normalized to a 1000-by-1000 scale. Its example action handler converts x using the screen width and y using the screen height, then executes the action. Your application supplies that execution environment; the response alone does not operate an arbitrary browser.
Google’s Computer Use guide: action execution and coordinate conversion
The documented conversion is x divided by 1000, multiplied by viewport width; y divided by 1000, multiplied by viewport height. The example handler converts the result to an integer. Match the dimensions to the viewport used by the handler rather than borrowing a monitor’s advertised resolution.
Work through a fictional click
Assume a fictional test response contains x=500 and y=250. Assume its target viewport is 1200 pixels wide and 800 pixels high. The horizontal calculation is 500/1000 × 1200 = 600. The vertical calculation is 250/1000 × 800 = 200. The resulting point is (600, 200).
Passing the original pair directly as pixels would instead click at (500, 250). In this invented example, that is 100 pixels left and 50 pixels below the converted point. A plausible-looking coordinate pair can therefore reach a different place when the handler uses the wrong units.
The example deliberately uses values that produce whole numbers. It is an arithmetic exercise, not a Gemini API response captured during testing. It establishes neither the correctness of a model’s chosen target nor the reliability of a real automation.
Make the first trial easy to inspect
Create a disposable demonstration page with a large target and a visible marker showing where a click landed. Record the chosen viewport width and height, the raw pair, the converted pair and the observed marker. These are proposed debugging records, not measurements collected for this guide.
Try another viewport size as a separate case. Confirm the page layout and dimensions for that case before judging its result. A changed layout may move the target as well as change the conversion, so do not assume the same raw pair remains the right instruction.
Separate two findings in the review: whether the arithmetic mapped the supplied pair correctly, and whether the selected point reached the intended control. Correct conversion can coexist with an incorrect target choice. Keeping those questions distinct makes a failed demonstration easier to diagnose.
Return the next observation
Google’s documented loop captures a fresh screenshot after execution and returns the resulting state for the next step. A successful arithmetic check is one component of that larger loop. This guide did not run an API request, execute browser actions or measure an agent’s success rate.
Check the Coordinate Mapping
