infrastructure-dom-reduce

Bounded structural reduction of HTML: keeps tags and the attributes selectors are built from, drops script/style/svg/noscript/template/iframe subtrees and comments entirely, and truncates every text node and kept attribute value. Two jobs at once — it makes a large page fit a small model's context, and it is the mechanism that keeps page CONTENT (conversation text, identifiers) out of whatever reads the result. Built on lol_html (Cloudflare's streaming HTML rewriter) rather than a hand-rolled parser.

What it is for

Bounded structural reduction of HTML.

# Why this exists

Two problems with one shape. A captured page is large — the live capture that motivated this crate was 2 MB, overwhelmingly CSS and inline data. A local 7B model has a few thousand tokens of usable context. And a page's text is the part that must not travel: conversation content, account identifiers, client material.

Both are solved by keeping the skeleton and discarding the flesh. Selectors are built from tag names and a small set of attributes; none of them need the text. So this keeps structure, truncates every text node and attribute value hard, and drops whole subtrees that are pure noise for selector purposes.

# Why lol_html

This estate does not hand-roll what a library already does (Gate 0.5). A first draft of this crate hand-rolled an HTML tokenizer; it was deleted unbuilt. lol_html (Cloudflare's streaming HTML rewriter, CSS-selector based) is the library crate: it already parses real-world HTML correctly, including cases a bounded tokenizer regex-walk gets wrong (nested quotes, self-closing rules, CDATA, encoding). Reduction happens as a rewrite pass — element/text handlers that drop or truncate — rather than a second parse.

# What "bounded" means here

Three caps, in the order they can fire. max_input_bytes (8 MB) cuts the input on a character boundary before the parser sees any of it, so no amount of pathological markup makes reduction unbounded in time; it leaves roughly 4x headroom over the 2 MB capture this crate was built for, because a backstop that fires on the design case is a working limit in disguise. Then:

The input is fed to the rewriter in fixed-size chunks rather than in one call. After every chunk, output size is checked against max_output_bytes; once it is met or exceeded, feeding stops without finishing the document — the remainder of a pathological or enormous input is never processed. A max_allowed_memory_usage backstop on the rewriter itself guards against a single buffered token (e.g. one absurdly long unterminated tag) blowing past the output cap before a chunk boundary is even reached. That backstop is configured to error, never to bail out gracefully: lol_html's graceful bail-out stops rewriting and passes the remaining input through verbatim — measured at 8 KB of a raw attribute value reaching the output, with nothing for reduce to observe. Any early stop — input cut, output cap, memory backstop, or a parser error — sets Reduction::truncated. A silent cap reads as "this is the whole page", which is how a model gets asked about markup it never saw.

# What it is not

Not a sanitizer. It reduces exposure by discarding, but a truncated text node or attribute value still carries its first characters. Treat the output as sensitive, just far less so than the input.

# Measured reduction

Against the generated page in tests/large_page.rs — 792 KB, deep nesting, over-long attributes, interleaved script/style — the default ReduceConfig cuts output to the 60 KB cap, a 92% reduction, dropping 251 elements while truncating 126 text nodes and 125 attribute values. Run the test to reproduce those figures; it prints them.

The input is generated rather than captured, deliberately: a committed "real page" fixture is where a browser capture carrying conversation content or a logged-in sidebar eventually lands. See the note at the top of that test.

No number is recorded here for a real 2 MB SPA capture — the workload this crate was built for. Such a page is expected to reduce further, since subtree-dropping and attribute-filtering carry more of the load when the mass is inline CSS and data rather than text. That remains an expectation, not a measurement, until one is measured.

Capabilities

crate root

Bounded structural reduction of HTML.

ReduceConfig

Bounded structural reduction of HTML.

Reduction

Bounded structural reduction of HTML.

How to use it

No examples/ target and no doctest in this crate's rustdoc. The tests listed under Verification are the closest executable usage.

Module structure

No public modules: the crate root is its whole surface.

Public surface

`crate root`

No pub use re-exports: every item above is declared in this crate.

Boundary

Depends on no other workspace tier.

Shares tier infrastructure with 82 other crates: infrastructure-acquire, infrastructure-adapters-google-calendar, infrastructure-adapters-google-gmail, infrastructure-adapters-google-places, infrastructure-adapters-google-trends, infrastructure-adapters-shodan, infrastructure-adapters-yelp, infrastructure-agent, … (82 total).

_What this crate deliberately does NOT own is a judgment. No committed registry records one for it, so none is stated here._

Where it sits

Dependencies

Runtime, from outside the workspace.

Development. None.

Build. None.

Depended on by. Nothing in this workspace.

Feature flags

No Cargo features are defined: every capability is unconditional, so no consumer can receive a half-wired crate.

Targets

Error model

No public error type was detected: no public item declares a type named *Error, and no public signature returns one.

Operational characteristics

No unsafe block, unsafe fn, unsafe impl or unsafe trait was found by the parser anywhere in this crate's source.

Configuration

No environment variable is read with a literal name anywhere in this crate. A variable whose key is computed at run time cannot be listed here, and is not claimed to be absent.

Related capabilities

No workspace crate depends on this one.

Verification

Evidence by module. How often each public module is named by something executable.

What the tests establish, by name:

• large_page_terminates_and_stays_under_cap — tests/large_page.rs
• a_long_script_body_never_leaks_even_a_truncated_fragment — src/lib.rs
• a_realistic_chat_page_keeps_the_selector_targets — src/lib.rs
• comments_are_dropped — src/lib.rs
• cut_input_lands_on_a_character_boundary — src/lib.rs
• cut_input_leaves_a_short_input_alone — src/lib.rs
• drops_noisy_attributes — src/lib.rs
• drops_script_and_style_bodies_entirely — src/lib.rs
• input_cap_is_reported_never_silent — src/lib.rs
• keeps_structure_and_selector_attributes — src/lib.rs
• long_attribute_values_are_truncated_and_reported — src/lib.rs
• memory_backstop_errors_rather_than_passing_input_through — src/lib.rs
• multibyte_text_is_never_split_mid_character — src/lib.rs
• output_cap_is_reported_never_silent — src/lib.rs
• short_text_is_kept_whole_so_structure_stays_readable — src/lib.rs
• truncated_text_does_not_grow_past_the_configured_max_when_raised — src/lib.rs
• truncates_text_nodes_this_is_the_privacy_lever — src/lib.rs
• unterminated_tag_does_not_hang_or_panic — src/lib.rs

Documentation coverage

pie showData
    title Public items with rustdoc
    "Documented" : 3
    "No rustdoc detected" : 1

Metrics

pie showData
    title Public API by kind
    "function" : 1
    "method" : 1
    "struct" : 2

pie showData
    title Rust source composition
    "Code" : 374
    "Blank or comment" : 195

Generation

Rendered by tools-corpus corpus readme from repository evidence alone, renderer schema 2, lexicon current. No model, network service or database was consulted. Regenerate with tools-corpus corpus readme --write; verify with --check.