> ## Documentation Index
> Fetch the complete documentation index at: https://yutori.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Navigator n1.5 (Browser Use)

**Navigator n1.5** is a browser-use model. It operates a webpage — given a task in natural language, the current screenshot, and the action history, it predicts the next browser action. Use the API model id `n1.5-latest` (or a dated version) in the `model` field of your `chat.completions` requests.

## Model Versions

| API model id | Description |
| - | - |
| `n1.5-latest` | Points to the latest stable Navigator n1.5 model. Currently points to `n1.5-20260428`. |
| `n1.5-20260428` | Stable release (2026-04-28). |

## Supported Actions

### Core Tools

The default tool set (`browser_tools_core-20260403`). 18 coordinate-based browser tools.

| Action | Description | Required Args | Optional Args |
| - | - | - | - |
| `left_click` | Left mouse click | `coordinates` | `ref`, `modifier` |
| `double_click` | Double left click | `coordinates` | `ref`, `modifier` |
| `triple_click` | Triple left click | `coordinates` | `ref`, `modifier` |
| `middle_click` | Middle mouse click | `coordinates` | `ref`, `modifier` |
| `right_click` | Right mouse click | `coordinates` | `ref`, `modifier` |
| `scroll` | Scroll in a direction | `coordinates`, `direction`, `amount` | `ref`, `modifier` |
| `type` | Type text into focused input | `text` | |
| `key_press` | Press a key or combination | `key` | |
| `drag` | Drag from start to end | `start_coordinates`, `coordinates` | |
| `mouse_move` | Move mouse to a point | `coordinates` | `ref` |
| `mouse_down` | Press and hold left mouse button | `coordinates` | `ref` |
| `mouse_up` | Release left mouse button | `coordinates` | `ref` |
| `go_back` | Browser back | | |
| `go_forward` | Browser forward | | |
| `wait` | Pause execution | | `duration` |
| `goto_url` | Navigate to URL | `url` | |
| `refresh` | Reload page | | |
| `hold_key` | Hold a key down | `key` | `duration` |

**Parameter notes:**

* `coordinates` is always `[x, y]` in the normalized 1000x1000 space.
* `ref` is an optional DOM element reference, used as an alternative to coordinates in browser contexts.
* `modifier` is a modifier key held during the action: `ctrl`, `shift`, `alt`, `meta`, `command`, or `super`.
* `direction` for scroll is one of: `down`, `up`, `left`, `right`.
* `amount` for scroll is an integer where 1 unit is approximately 10% of the screen height.

### Expanded Browser Tools

Includes all core tools plus DOM/ref-based extras (`browser_tools_expanded-20260403`):

| Action | Description | Required Args | Optional Args |
| - | - | - | - |
| `extract_elements` | Extract a structured ARIA-snapshot-style representation of the page's interactive and semantic elements. Returns a `pageContent` string of `- role "name" [ref=ref_N]` lines (with optional `id`, `href` attributes) that the model can read. Each `ref` is a stable handle that later actions (`left_click`, `set_element_value`, …) can target. | | `filter` (`visible`, `interactive`, `all`) |
| `find` | Search the page for elements whose ARIA-snapshot line matches a substring. Returns a `matches` list of `- role "name" [ref=ref_N]` lines (and a `totalMatches` count) so the model can target them. | `text` | |
| `set_element_value` | Set the value of an `<input>`, `<textarea>`, or `<select>` element directly by ref, dispatching the right `input`/`change` events so the page sees the update. | `ref`, `value` | |
| `execute_js` | Run an arbitrary JavaScript expression or statement block against the page and return the (JSON-serialized) result back to the model. Lets the model interact with the page directly when that's faster or more reliable than the equivalent click/type/scroll sequence — reading hidden state, calling page-internal APIs, or scripting multi-step flows in one shot. | `text` | |

> [!NOTE]
>
> Like every Navigator tool, these are predicted by the model and executed by your client — the API never touches your browser. What's specific to the expanded tools is **how** you execute them: instead of mapping to a Playwright primitive (click, type, scroll), each one needs custom JavaScript evaluated against the page (e.g., via `page.evaluate()`). The result comes back as a `tool` message in the next request.

#### Reference Implementation

The [Yutori Python SDK](https://github.com/yutori-ai/yutori-sdk-python) bundles a reference implementation for each tool, along with an async helper that evaluates them against a Playwright page. Import the script constants and `evaluate_tool_script` from `yutori.navigator.tools`:

| Tool | SDK constant | Reference script | Returns |
| - | - | - | - |
| `extract_elements` | `EXTRACT_ELEMENTS_SCRIPT` | [extract\_elements.js][s-extract] | `{success, pageContent}` — also stores live refs on `window.__yutoriElementRefs` for later ref-based actions |
| `find` | `FIND_SCRIPT` | [find.js][s-find] | `{success, matches, totalMatches}` — substring filter over the same DOM walk |
| `set_element_value` | `SET_ELEMENT_VALUE_SCRIPT` | [set\_element\_value.js][s-set] | `{success, message}` — sets the value by `ref` and dispatches the right `input`/`change` events |
| `execute_js` | `EXECUTE_JS_SCRIPT` | [execute\_js.js][s-exec] | `{success, hasResult, result}` — wraps the model's snippet in an `AsyncFunction` so both expressions and statement blocks work |
| `left_click` / `scroll` via `ref` | `GET_ELEMENT_BY_REF_SCRIPT` | [get\_element\_by\_ref.js][s-getref] | `{success, coordinates}` — resolves a `ref` to viewport pixel coordinates and scrolls it into view |

[s-extract]: https://github.com/yutori-ai/yutori-sdk-python/blob/main/yutori/navigator/tools/js/extract_elements.js

[s-find]: https://github.com/yutori-ai/yutori-sdk-python/blob/main/yutori/navigator/tools/js/find.js

[s-set]: https://github.com/yutori-ai/yutori-sdk-python/blob/main/yutori/navigator/tools/js/set_element_value.js

[s-exec]: https://github.com/yutori-ai/yutori-sdk-python/blob/main/yutori/navigator/tools/js/execute_js.js

[s-getref]: https://github.com/yutori-ai/yutori-sdk-python/blob/main/yutori/navigator/tools/js/get_element_by_ref.js

The helper `evaluate_tool_script(page, SCRIPT, *args)` JSON-serializes its arguments, evaluates the script against the page, and returns a Python `dict`. Three common patterns:

```python theme={null}
from yutori.navigator.tools import (
    EXECUTE_JS_SCRIPT,
    EXTRACT_ELEMENTS_SCRIPT,
    GET_ELEMENT_BY_REF_SCRIPT,
    evaluate_tool_script,
)

# 1. Read structured DOM for the model — feed `pageContent` back as the tool result.
result = await evaluate_tool_script(page, EXTRACT_ELEMENTS_SCRIPT, "visible")
tool_result_text = result["pageContent"]

# 2. Resolve a `ref` (from extract_elements/find) into viewport pixels before a click.
result = await evaluate_tool_script(page, GET_ELEMENT_BY_REF_SCRIPT, ref)
if result["success"]:
    px_x, px_y = result["coordinates"]

# 3. Run the model's `execute_js` snippet — pass the raw `text` argument; the script
#    already wraps it in an async IIFE. Surface `result` when hasResult is True.
result = await evaluate_tool_script(page, EXECUTE_JS_SCRIPT, args["text"])
tool_result_text = str(result["result"]) if result.get("hasResult") else "undefined"
```

For the full agent loop — including how each tool's response envelope feeds back into the next assistant turn — see [`examples/navigator_n1_5.py`](https://github.com/yutori-ai/yutori-sdk-python/blob/main/examples/navigator_n1_5.py) in the SDK.

### Key Space

Navigator n1.5 uses lowercase key names. Combinations are joined with `+`, and sequential presses are separated by spaces.

| Category | Key Names |
| - | - |
| Modifiers | `ctrl`, `alt`, `shift`, `meta`, `command`, `super` |
| Common | `enter`, `backspace`, `delete`, `tab`, `esc`, `space` |
| Arrow keys | `left`, `right`, `up`, `down` |
| Page navigation | `pageup`, `pagedown`, `home`, `end` |
| Function keys | `f1` through `f12` |

Examples: `ctrl+c`, `ctrl+shift+t`, `alt+left`, `down down down enter`

## Coordinate System

n1.5 outputs **normalized coordinates in a 1000×1000 space**, origin top-left. Scale them to the real pixel size of the browser viewport before executing an action — the model is never told the resolution.

```python theme={null}
from yutori.navigator import denormalize_coordinates

pixel_x, pixel_y = denormalize_coordinates(
    coordinates=[500, 465],
    width=viewport_width,
    height=viewport_height,
)
```

See [`yutori.navigator.coordinates`](https://github.com/yutori-ai/yutori-sdk-python/blob/main/yutori/navigator/coordinates.py) for the inverse `normalize_coordinates` function.

## Features

### Tool Sets

Use the `tool_set` parameter to select which set of browser tools are available to the model:

```python theme={null}
response = client.chat.completions.create(
    model="n1.5-latest",
    messages=[...],
    extra_body={
        "tool_set": "browser_tools_expanded-20260403",
    }
)
```

Available tool sets:

* **`browser_tools_core-20260403`** (default) — coordinate-based visual browser tools
* **`browser_tools_expanded-20260403`** — core + DOM-based tools (`extract_elements`, `find`, `set_element_value`, `execute_js`)

### Disabling Specific Tools

Remove specific tools from the active tool set:

```python theme={null}
response = client.chat.completions.create(
    model="n1.5-latest",
    messages=[...],
    extra_body={
        "disable_tools": ["hold_key", "drag"],
    }
)
```

### JSON Structured Output

Provide a `json_schema` to get structured data extracted from the model's response. The schema is appended to your task message, and the model returns JSON inside ` ```json ` code fences. The API parses this and returns it as a `parsed_json` field.

```python theme={null}
response = client.chat.completions.create(
    model="n1.5-latest",
    messages=[...],
    extra_body={
        "json_schema": {
            "type": "object",
            "properties": {
                "product_name": {"type": "string"},
                "price": {"type": "number"}
            },
            "required": ["product_name", "price"]
        }
    }
)

# Access the parsed result
parsed = response.parsed_json  # {"product_name": "Widget Pro", "price": 29.99}
```

When `json_schema` is provided, the API also adds a structural tag for guided decoding of the JSON output, constraining it to match your schema.

If the model doesn't return valid JSON (e.g., it's still navigating), the `parsed_json` field will not be present in the response.

## Multi-Turn Conversations

The Python SDK provides `create_trimmed` / `acreate_trimmed` (in `yutori.navigator`) which strip older screenshots automatically while preserving recent ones and all text — by default keeping the **6 most recent** screenshots (and always the latest), and only trimming when a request would otherwise exceed the size limit.

Do not drop messages — the model uses its own action history to avoid repeating work. Keep at least the **2 most recent screenshots**, and never send a request with no screenshot at all; stale-only or screenshot-free requests sharply degrade grounding and can make the model loop.

Include the assistant's previous response with its `tool_calls`, followed by tool results with the new screenshot:

```python theme={null}
response = client.chat.completions.create(
    model="n1.5-latest",
    messages=[
        # Initial user message with task and first screenshot
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Search for Yutori on Google."},
                {"type": "image_url", "image_url": {"url": f"data:image/webp;base64,{screenshot_1}"}}
            ]
        },
        # Assistant's response with tool call
        {
            "role": "assistant",
            "content": "I can see the Google homepage. I'll click on the search bar.",
            "tool_calls": [
                {
                    "id": "chatcmpl-tool-123",
                    "type": "function",
                    "function": {
                        "name": "left_click",
                        "arguments": "{\"coordinates\": [500, 465]}"
                    }
                }
            ]
        },
        # Tool result with current URL and new screenshot
        {
            "role": "tool",
            "tool_call_id": "chatcmpl-tool-123",
            "content": [
                {"type": "text", "text": "Clicked 1x with left\nCurrent URL: https://www.google.com/"},
                {"type": "image_url", "image_url": {"url": f"data:image/webp;base64,{screenshot_2}"}}
            ]
        }
    ]
)
```

The tool result can be a short description of what the tool did or whether it was successful. Including the current URL in the tool response is important for better attribution of information sources.

If the tool extracts additional information (e.g. `extract_elements`, `execute_js`), the raw output of the tool can be provided as the tool result so the model also has visibility of the extracted information.

## Screenshot Requirements

Screenshots should capture only the browser content itself. Do **not** include the operating system UI, window title bars, browser tabs, URL bars, or other chrome elements.

For the best performance, render screenshots in **WXGA (1280×800, 16:10)**. The model should generalize well to most other resolutions, but grounding accuracy may degrade with extreme aspect ratios.

We recommend using the **WebP** format for screenshots, as it offers significantly better compression than PNG — especially for multi-step trajectories with many images.

The [Python SDK](https://github.com/yutori-ai/yutori-sdk-python) provides helpers that handle resizing, WebP conversion, and base64 encoding:

```bash theme={null}
pip install yutori
```

```python theme={null}
from yutori.navigator import screenshot_to_data_url, playwright_screenshot_to_data_url

# From raw screenshot bytes (any format)
image_url = screenshot_to_data_url(screenshot_bytes)

# Or directly from a Playwright page
image_url = playwright_screenshot_to_data_url(page)

# Async variant also available
# image_url = await aplaywright_screenshot_to_data_url(page)
```

See [`yutori.navigator.images`](https://github.com/yutori-ai/yutori-sdk-python/blob/main/yutori/navigator/images.py) for full options (custom resolution, quality settings).

## Custom Tools

You can provide additional tools alongside the built-in browser actions using the `tools` parameter. Custom tools are appended after the default tool set.

```python theme={null}
response = client.chat.completions.create(
    model="n1.5-latest",
    messages=[...],
    tools=[
        {
            "type": "function",
            "function": {
                "name": "extract_content_and_links",
                "description": "Extracts page content and hyperlinks relevant to the user task. This operation is strictly read-only and never interacts with or alters the page.",
                "parameters": {"type": "object", "properties": {}, "required": []}
            }
        }
    ]
)
```

Results of custom tool calls should also be provided as a tool response so the model can use it for subsequent decisions:

```python theme={null}
{
    "role": "tool",
    "tool_call_id": "chatcmpl-tool-456",
    "content": [
        {
            "type": "text",
            "text": "Visible buttons and links:\n- About https://about.google/\n- Store https://store.google.com/\n- Gmail https://mail.google.com/\nCurrent page URL: https://www.google.com/"
        },
        {
            "type": "image_url",
            "image_url": {"url": f"data:image/webp;base64,{screenshot}"}
        }
    ]
}
```

## Tool Choice

Control whether tool calls are parsed into the `tool_calls` array:

* `"auto"` (default): Parses and returns tool calls as a structured `tool_calls` list
* `"none"`: Returns the raw model response as content text (tool calls may still appear inside `<tool_call>` XML tags in content)

## Structured Decoding

By default, the API uses a `structural_tag` response format to enforce valid tool call generation via guided decoding. You do **not** need to provide this yourself — the API generates it automatically based on the active model and tool set. If custom tools are included in your request, their schemas are automatically incorporated.

We do not recommend overriding the `response_format` unless you also set `tool_choice="none"` to work with the raw model output directly.

## Prompting

A system message you supply adds to n1.5's defaults rather than replacing them, and extra behavioral instructions generally degrade results — we do not recommend a custom system prompt.

Put task-specific instructions in the **first user message**, after the task description:

```json theme={null}
{
  "messages": [
    {
      "role": "user",
      "content": "Click the login button. Additional instructions: be careful with popups."
    }
  ]
}
```

Avoid interrupting a trajectory with extra user messages, except to stop the model and have it summarize:

```python theme={null}
messages.append({
    "role": "user",
    "content": f"Stop here. Summarize your current progress and list all findings relevant to: {task}"
})
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.