tools-gui-harness

Native desktop GUI test harness: find/focus a window (composes infrastructure-screencast), synthesize real hardware mouse/keyboard input via xdotool (physical focus, not synthetic events), screenshot via infrastructure-screen-capture, and read rendered text back via infrastructure-ocr-tesseract. For driving/verifying a real running desktop app (egui/eframe, GTK, Qt, ...) end to end, the same way a human would.

What it is for

Native desktop GUI test harness: find/focus a window, drive it with real hardware mouse/keyboard input, screenshot it, and read its rendered text back via OCR. For verifying a real running desktop app (egui/eframe, GTK, Qt, ...) end to end, the way a human tester would -- not a mock, not a synthetic-event framework.

Composes existing library crates rather than re-implementing them (Gate 0.5): window find/focus is infrastructure-screencast, screenshot is infrastructure-screen-capture, OCR is infrastructure-ocr-tesseract. The one genuinely new piece is input -- hardware mouse/keyboard actuation via xdotool, which existed before only as a private helper inside tools-oauth-consent (Gate 4.5 follow-up: that crate should migrate to this one).

Capabilities

capture (other)

Screenshot capture, thin over infrastructure-screen-capture.

CaptureError

Screenshot capture, thin over infrastructure-screen-capture.

Shot

Screenshot capture, thin over infrastructure-screen-capture.

Input

Hardware mouse/keyboard actuation via xdotool (argv subprocess, never a

InputError

Hardware mouse/keyboard actuation via xdotool (argv subprocess, never a

VerifiedClickError

Hardware mouse/keyboard actuation via xdotool (argv subprocess, never a

OcrHarnessError

Read rendered text back out of a screenshot, via infrastructure-ocr-tesseract.

WindowError

Window find/focus: composes infrastructure-screencast::window rather than

How to use it

From `examples/guarded_click.rs`:

use std::env;
use tools_gui_harness::input::Input;

fn main() {
    let mut args = env::args().skip(1);
    let x: i64 = args
        .next()
        .expect("usage: guarded_click <x> <y> <expected-title-substring>")
        .parse()
        .expect("x must be an integer");
    let y: i64 = args
        .next()
        .expect("missing y")
        .parse()
        .expect("y must be an integer");
    let expected = args.next().expect("missing expected-title-substring");

    match Input::new().verified_click(x, y, 1, &expected) {

From `examples/smoke.rs`:

use tools_gui_harness::{capture, ocr};

fn main() {
    let shot = capture::capture().expect("capture failed");
    println!(
        "captured {}x{} ({} PNG bytes)",
        shot.width,
        shot.height,
        shot.png.len()
    );

    std::fs::write("/tmp/gui-harness-smoke.png", &shot.png).expect("write failed");
    println!("wrote /tmp/gui-harness-smoke.png");

    let text = ocr::recognize_png(&shot.png, shot.width, shot.height).expect("ocr failed");
    println!("OCR recognized {} chars", text.len());
    println!("--- first 300 chars ---");
    println!("{}", text.chars().take(300).collect::<String>());

Module structure

tools_gui_harness

• capture
• input
• ocr
• window

flowchart TD
  n_tools_gui_harness["tools_gui_harness"]
  n_tools_gui_harness --> n_capture["capture"]
  n_tools_gui_harness --> n_input["input"]
  n_tools_gui_harness --> n_ocr["ocr"]
  n_tools_gui_harness --> n_window["window"]

Public surface

`capture`

`input`

`ocr`

`window`

No pub use re-exports: every item above is declared in this crate.

Boundary

Reaches into infrastructure.

Shares tier tools with 84 other crates: tools-advisory-reach, tools-archive-guard, tools-artifact-scaffold, tools-ask-ai-core, tools-ask-ais, tools-ask-gemini, tools-book, tools-book-report, … (84 total).

_What this crate deliberately does NOT own is a judgment. No committed registry records one for it, so none is stated here._

Where it sits

Tier flow. Which tiers this crate's own edges cross.

flowchart LR
  n_tools["tools"] --> n_infrastructure["infrastructure"]

Dependencies

Runtime, in this workspace.

Runtime, from outside the workspace.

Development. None.

Build. None.

Depended on by. 1 workspace crate.

Signal flow — what reaches this crate, and what it reaches.

flowchart LR
  n_tools_demo_systats["tools-demo-systats"] -->|uses| SELF
  SELF["tools-gui-harness"]
  SELF -->|runtime| n_infrastructure_ocr_engine["infrastructure-ocr-engine"]
  SELF -->|runtime| n_infrastructure_ocr_tesseract["infrastructure-ocr-tesseract"]
  SELF -->|runtime| n_infrastructure_screen_capture["infrastructure-screen-capture"]
  SELF -->|runtime| n_infrastructure_screencast["infrastructure-screencast"]
  classDef self fill:#1f883d,stroke:#1f883d,color:#fff;
  class SELF self;

Feature flags

No Cargo features are defined: every capability is unconditional, so no consumer can receive a half-wired crate.

Targets

Error model

Operational characteristics

No unsafe block, unsafe fn, unsafe impl or unsafe trait was found by the parser anywhere in this crate's source.

Configuration

No environment variable is read with a literal name anywhere in this crate. A variable whose key is computed at run time cannot be listed here, and is not claimed to be absent.

Related capabilities

1 workspace crate depends on this one: tools-demo-systats.

Verification

Evidence by module. How often each public module is named by something executable.

What the tests establish, by name:

• boundary_title_matches_empty_expected_substring_always_matches — src/input.rs
• deny_missing_binary_propagates_from_click_and_settle — src/input.rs
• deny_missing_binary_propagates_from_key — src/input.rs
• deny_missing_binary_propagates_from_move_mouse — src/input.rs
• deny_missing_binary_propagates_from_type_text — src/input.rs
• deny_missing_binary_yields_spawn_error_not_a_panic — src/input.rs
• deny_title_matches_rejects_a_different_window — src/input.rs
• deny_verified_click_missing_binary_never_reaches_the_click — src/input.rs
• valid_click_args_are_mousemove_then_click — src/input.rs
• valid_key_args_use_the_double_dash_separator — src/input.rs
• valid_move_args_are_mousemove_only — src/input.rs
• valid_title_matches_is_case_insensitive_substring — src/input.rs
• valid_type_args_pass_text_verbatim_after_separator — src/input.rs
• deny_active_window_title_missing_binary_is_an_error — src/window.rs
• deny_active_window_title_nonzero_exit_is_an_error — src/window.rs
• valid_active_window_title_returns_trimmed_stdout — src/window.rs

Documentation coverage

pie showData
    title Public items with rustdoc
    "Documented" : 12
    "No rustdoc detected" : 9

Metrics

pie showData
    title Public API by kind
    "enum" : 5
    "function" : 5
    "method" : 9
    "struct" : 2

pie showData
    title Rust source composition
    "Code" : 341
    "Blank or comment" : 127

Generation

Rendered by tools-corpus corpus readme from repository evidence alone, renderer schema 2, lexicon current. No model, network service or database was consulted. Regenerate with tools-corpus corpus readme --write; verify with --check.