tools tier

`tools-dom-shard-extract`

Reduce raw HTML forms/links to the minimal structural shard needed for one-time AI classification (ADR 0035 'Shard construction'): method/path-depth/hidden-field-category/label for forms, path-depth/query/fragment/microdata for links -- never site names, usernames, tokens, or link text.

Reduce raw HTML forms/links to the minimal structural shard needed for one-time AI classification (ADR 0035 'Shard construction'): method/path-depth/hidden-field-category/label for forms, path-depth/query/fragment/microdata for links -- never site names, usernames, tokens, or link text.

Tier	tools
Role	unclassified (baselined)
Path	`crates/tools/dom-shard-extract`
Edition	2021
Targets	tools-dom-shard-extract, tools_dom_shard_extract
Public items	30 across 1 module
Tests	14

What it is for

Reduce raw HTML forms/links to the smallest structural shape needed for one-time AI classification -- ADR 0035 ("Shard construction"). Pipeline: raw HTML -> normalize (lol_html, structural only) -> group structurally-identical forms/links -> emit a classify_interactive_pattern shard. Never discloses site names, usernames, link/button text content (form submit labels are the one deliberate exception -- see below), hidden field names, or token values; only their structural shape.

Link-shape extraction is the "enumerate the list" step of the multi-step traversal loop worked example in ADR 0035: distinguishing a resource link (repo, item, record) from navigation chrome (nav bar, footer, skip-links) by structure -- path depth, query/fragment shape, and itemprop microdata -- never by href text or link text. Container-position context (an ADR phrase like "inside a repo-list row") is a known, not-yet-built extension -- this uses only signals present on the tag itself.

Capabilities

`crate root`

Reduce raw HTML forms/links to the smallest structural shape needed for

Item
`fn classify_hidden_field(name : & str) -> String`
`fn action_shape(action : & str) ->(usize, bool)`
`fn link_shape(href : & str) ->(usize, bool, bool)`
`fn dedup_with_counts <T : PartialEq + Clone>(items : & T) -> Vec <(T, usize)>`
`pub const FORM_ALLOWED_OUTPUTS: & str; 4`
`pub const LINK_ALLOWED_OUTPUTS: & str; 3`

`ClassifyShard`

Reduce raw HTML forms/links to the smallest structural shape needed for

Item
`pub struct ClassifyShard`
`ClassifyShard :: fn for_form(form : & ObservedForm, count : usize) -> Self`

`LinkClassifyShard`

Reduce raw HTML forms/links to the smallest structural shape needed for

Item
`pub struct LinkClassifyShard`
`LinkClassifyShard :: fn for_link(link : & ObservedLink, count : usize) -> Self`

`LinkNodeShape`

Reduce raw HTML forms/links to the smallest structural shape needed for

Item
`pub struct LinkNodeShape`

`NodeShape`

Reduce raw HTML forms/links to the smallest structural shape needed for

Item
`pub struct NodeShape`

`ObservedForm`

Reduce raw HTML forms/links to the smallest structural shape needed for

Item
`pub struct ObservedForm`
`fn extract_forms(html : & str) -> Vec <ObservedForm>`

`ObservedLink`

Reduce raw HTML forms/links to the smallest structural shape needed for

Item
`pub struct ObservedLink`
`fn extract_links(html : & str) -> Vec <ObservedLink>`

`rule_store (other)`

The persistence layer named as missing scope in ADR 0035 ("The pipeline

Item
`fn shape_key <T : Serialize>(kind : & str, shape : & T) -> String`

`CandidateRule`

The persistence layer named as missing scope in ADR 0035 ("The pipeline

Item
`pub struct CandidateRule`
`CandidateRule :: fn validate(self, observed_transition : impl Into <String>,) -> Result <ValidatedRule, RejectedCandidate>`

`RejectedCandidate`

The persistence layer named as missing scope in ADR 0035 ("The pipeline

Item
`pub struct RejectedCandidate`

`RuleStore`

The persistence layer named as missing scope in ADR 0035 ("The pipeline

Item
`pub struct RuleStore`
`RuleStore :: fn new() -> Self`
`RuleStore :: fn load(path : & Path) -> io::Result <Self>`
`RuleStore :: fn save(& self, path : & Path) -> io::Result <()>`
`RuleStore :: fn lookup(& self, key : & str) -> Option <& ValidatedRule>`
`RuleStore :: fn insert(& mut self, key : String, rule : ValidatedRule)`
`RuleStore :: fn len(& self) -> usize`
`RuleStore :: fn is_empty(& self) -> bool`

`ValidatedRule`

The persistence layer named as missing scope in ADR 0035 ("The pipeline

Item
`pub struct ValidatedRule`

`ValidationEvidence`

The persistence layer named as missing scope in ADR 0035 ("The pipeline

Item
`pub struct ValidationEvidence`

How to use it

No examples/ target and no doctest in this crate's rustdoc. The tests listed under Verification are the closest executable usage.

Module structure

tools_dom_shard_extract

rule_store

Public surface

`crate root`

Item	What it is
`pub struct ObservedForm`	One `<form>` as deterministically observed -- structure only, no real identifiers
`pub struct ObservedLink`	One `<a>` as deterministically observed -- structure only
`fn classify_hidden_field(name : & str) -> String`	—
`fn action_shape(action : & str) ->(usize, bool)`	—
`fn link_shape(href : & str) ->(usize, bool, bool)`	—
`fn extract_forms(html : & str) -> Vec <ObservedForm>`	Split raw HTML into per-`<form>` fragments and parse each independently with `lol_html`
`fn extract_links(html : & str) -> Vec <ObservedLink>`	Split raw HTML into per-`<a>` fragments, same naive non-nesting flush as `extract_forms` -- anchors don't nest inside other anchors in practice, so "first `</a>` after this `<a` " is a safe boundary here too.
`fn dedup_with_counts <T : PartialEq + Clone>(items : & T) -> Vec <(T, usize)>`	Group structurally-identical items -- this IS the "does removing it change the answer" minimality test applied mechanically: two items with the same shape are the same pattern candidate, so only one shard is needed for all of them, plus a count.
`pub struct ClassifyShard`	—
`pub struct NodeShape`	—
`pub const FORM_ALLOWED_OUTPUTS: & str; 4`	—
`ClassifyShard :: fn for_form(form : & ObservedForm, count : usize) -> Self`	—
`pub struct LinkClassifyShard`	—
`pub struct LinkNodeShape`	—
`pub const LINK_ALLOWED_OUTPUTS: & str; 3`	—
`LinkClassifyShard :: fn for_link(link : & ObservedLink, count : usize) -> Self`	—

`rule_store`

Item	What it is
`pub struct CandidateRule`	An AI's hypothesis for a shape -- proposed, not yet proven
`CandidateRule :: fn validate(self, observed_transition : impl Into <String>,) -> Result <ValidatedRule, RejectedCandidate>`	—
`pub struct RejectedCandidate`	A candidate whose observed transition did not match its expectation -- this is a Repair signal (ADR 0035 "Discover / Learn / Replay / Repair"), not a rule, and cannot be inserted.
`pub struct ValidationEvidence`	Proof that a `CandidateRule`'s expected transition was actually observed
`pub struct ValidatedRule`	A `CandidateRule` plus proof its expected transition was actually observed
`fn shape_key <T : Serialize>(kind : & str, shape : & T) -> String`	A stable string key for any structural shape
`pub struct RuleStore`	Shape-keyed lookup table
`RuleStore :: fn new() -> Self`	—
`RuleStore :: fn load(path : & Path) -> io::Result <Self>`	Loads a store from `path`
`RuleStore :: fn save(& self, path : & Path) -> io::Result <()>`	—
`RuleStore :: fn lookup(& self, key : & str) -> Option <& ValidatedRule>`	—
`RuleStore :: fn insert(& mut self, key : String, rule : ValidatedRule)`	Only accepts a `ValidatedRule` -- there is no overload that takes a `CandidateRule`
`RuleStore :: fn len(& self) -> usize`	—
`RuleStore :: fn is_empty(& self) -> bool`	—

Re-exports. Exported here, defined elsewhere.

Export	Defined in
`{shape_key,CandidateRule,RejectedCandidate,RuleStore,ValidatedRule,ValidationEvidence,}`	`rule_store::{shape_key,CandidateRule,RejectedCandidate,RuleStore,ValidatedRule,ValidationEvidence,}`

Boundary

Reaches into infrastructure.

Shares tier tools with 84 other crates: tools-advisory-reach, tools-archive-guard, tools-artifact-scaffold, tools-ask-ai-core, tools-ask-ais, tools-ask-gemini, tools-book, tools-book-report, … (84 total).

_What this crate deliberately does NOT own is a judgment. No committed registry records one for it, so none is stated here._

Where it sits

Tier (ontology)	`tools`
Architectural role (taxonomy)	`unclassified (baselined)`
Location	`crates/tools/dom-shard-extract`
Vocabulary in force (lexicon)	`current`

Tier flow. Which tiers this crate's own edges cross.

flowchart LR
  n_tools["tools"] --> n_infrastructure["infrastructure"]

Dependencies

Runtime, in this workspace.

Crate	Tier	Optional	Only on
`infrastructure-fetcher`	infrastructure	no	always

Runtime, from outside the workspace.

Crate	Requirement	Features	Optional	Only on
`lol_html`	`^3.0.1`	—	no	always
`serde`	`^1`	`derive`	no	always
`serde_json`	`^1`	—	no	always
`tokio`	`^1`	`full`	no	always
`url`	`^2`	—	no	always

Development. None.

Build. None.

Depended on by. Nothing in this workspace.

Signal flow — what reaches this crate, and what it reaches.

flowchart LR
  SELF["tools-dom-shard-extract"]
  SELF -->|runtime| n_infrastructure_fetcher["infrastructure-fetcher"]
  classDef self fill:#1f883d,stroke:#1f883d,color:#fff;
  class SELF self;

Feature flags

No Cargo features are defined: every capability is unconditional, so no consumer can receive a half-wired crate.

Targets

Kind	Name	Source
bin	`tools-dom-shard-extract`	`src/main.rs`
lib	`tools_dom_shard_extract`	`src/lib.rs`

Error model

No public error type was detected: no public item declares a type named *Error, and no public signature returns one.

Operational characteristics

Property	Evidence
async public surface	none detected
async runtime	yes
database access	none detected
network I/O	none detected
unsafe code	none detected
environment variables	none detected

No unsafe block, unsafe fn, unsafe impl or unsafe trait was found by the parser anywhere in this crate's source.

Configuration

No environment variable is read with a literal name anywhere in this crate. A variable whose key is computed at run time cannot be listed here, and is not claimed to be absent.

No workspace crate depends on this one.

Verification

Kind	Count
Unit tests	14
Integration tests	0
Examples	0
Doctests	0

Evidence by module. How often each public module is named by something executable.

Module	Tests	Examples	Consumers
`crate root`	14	0	0
`rule_store`	6	0	0

What the tests establish, by name:

action_shape_counts_segments_and_detects_query_target — src/lib.rs
button_submit_text_content_populates_label_same_as_input_value — src/lib.rs
classify_hidden_field_names_tokens_and_leaves_others_opaque — src/lib.rs
dedup_collapses_identical_shapes_and_counts_them — src/lib.rs
link_shape_detects_fragment_and_query_independently — src/lib.rs
resource_link_vs_navigation_chrome_distinguished_by_microdata_and_fragment — src/lib.rs
shard_json_never_contains_raw_token_or_href_text — src/lib.rs
load_missing_file_returns_empty_store — src/rule_store.rs
matching_observation_validates_the_candidate — src/rule_store.rs
mismatched_observation_rejects_the_candidate_instead_of_validating — src/rule_store.rs
only_validated_rules_are_insertable_a_candidate_cannot_be — src/rule_store.rs
save_then_load_roundtrips_validated_rules — src/rule_store.rs
shape_key_is_stable_for_identical_shapes_and_differs_for_different_ones — src/rule_store.rs
validated_rule_stamps_a_plausible_unix_timestamp — src/rule_store.rs

Documentation coverage

Measure	Documented	Total
Public items with rustdoc	13	30
Public modules with a `//!` block	1	1

pie showData
    title Public items with rustdoc
    "Documented" : 13
    "No rustdoc detected" : 17

Metrics

Metric	Value
Rust source files	3
Source lines	892
Code lines	658
Public API items	30
Public modules	1
Tests	14
Examples	0
Cargo features	0
Direct runtime dependencies	6
Workspace reverse dependencies	0

pie showData
    title Public API by kind
    "constant" : 2
    "function" : 7
    "method" : 10
    "struct" : 11

pie showData
    title Rust source composition
    "Code" : 658
    "Blank or comment" : 234

Generation

Rendered by tools-corpus corpus readme from repository evidence alone, renderer schema 2, lexicon current. No model, network service or database was consulted. Regenerate with tools-corpus corpus readme --write; verify with --check.

All tools · Manual