<img>/<link>/OG-image URL discovery from already-fetched HTML -- no storage, no download, no archival (Gate 0.5 rejected the source's HtmlArchive storage layer as an infrastructure-storage duplicate; this crate is the net-new discovery half only)
| Tier | infrastructure |
| Role | unclassified (baselined) |
| Path | crates/infrastructure/web-asset-discoverer |
| Edition | 2021 |
| Targets | infrastructure_web_asset_discoverer |
| Public items | 6 across 1 module |
| Tests | 15 |
What it is for
infrastructure-web-asset-discoverer -- <img>/<link>/OG-image URL discovery from already-fetched HTML, ported from plumber/src/extraction/discovery.rs::AssetDiscoverer.
This crate does exactly one thing: given an HTML document and its base URL, produce the list of asset URLs (<img src>/<img srcset>/ <picture><source srcset>, PDF/document <a href>, favicon/apple-touch <link rel>, and Open Graph / Twitter Card <meta> images) referenced by that document. It does not fetch, download, or store anything -- Gate 0.5 dedup (docs/userstories/sprint-3.46-plumber-harvest.md, crate table row 13) rejected the source's HtmlArchive storage layer (storage/html_archive.rs) as a duplicate of infrastructure-storage's date-sharded, content-addressed architecture. AssetDiscoverer itself -- the part of discovery.rs that parses URLs out of markup -- has no existing counterpart and is this crate's entire scope. The source file contained no download/storage code of its own (it only ever produced DiscoveredAsset { url, .. } values for some other module to act on), so this is a whole-file lift, not a partial extraction.
Not a foundation_web_analysis::Analyzer implementor, unlike most of its wave-2/3 siblings -- like infrastructure-web-contact-extractor, this is a standalone parsing utility. It takes HTML a caller already fetched (by whatever means -- typically infrastructure-fetcher's SafeHttpClient, the repo's SSRF-hardened client) and has no HTTP client dependency of its own.
Capabilities
AssetDiscoverer
<img>/<link>/OG-image URL discovery, ported from
| Item |
|---|
pub struct AssetDiscoverer |
AssetDiscoverer :: fn discover(html : & str, base_url : & Url) -> AssetDiscovery |
AssetDiscovery
<img>/<link>/OG-image URL discovery, ported from
| Item |
|---|
pub struct AssetDiscovery |
AssetKind
<img>/<link>/OG-image URL discovery, ported from
| Item |
|---|
pub enum AssetKind |
AssetKind :: fn fmt(& self, f : & mut fmt::Formatter <'_>) -> fmt::Result |
DiscoveredAsset
<img>/<link>/OG-image URL discovery, ported from
| Item |
|---|
pub struct DiscoveredAsset |
How to use it
No examples/ target and no doctest in this crate's rustdoc. The tests listed under Verification are the closest executable usage.
Module structure
infrastructure_web_asset_discoverer
discoverer
Public surface
`discoverer`
| Item | What it is |
|---|---|
pub enum AssetKind | The kind of asset a DiscoveredAsset resolves to |
AssetKind :: fn fmt(& self, f : & mut fmt::Formatter <'_>) -> fmt::Result | Renders the source's original asset_type string, for callers that want the label rather than the enum (logging, serialization). |
pub struct DiscoveredAsset | One asset URL discovered in an HTML document. |
pub struct AssetDiscovery | Result of one AssetDiscoverer::discover pass. |
pub struct AssetDiscoverer | Discovers assets (images, PDFs, documents) referenced by an HTML document |
AssetDiscoverer :: fn discover(html : & str, base_url : & Url) -> AssetDiscovery | Extract all asset URLs from html, resolved against base_url. |
Re-exports. Exported here, defined elsewhere.
| Export | Defined in |
|---|---|
{AssetDiscoverer,AssetDiscovery,AssetKind,DiscoveredAsset} | discoverer::{AssetDiscoverer,AssetDiscovery,AssetKind,DiscoveredAsset} |
Boundary
Depends on no other workspace tier.
Shares tier infrastructure with 82 other crates: infrastructure-acquire, infrastructure-adapters-google-calendar, infrastructure-adapters-google-gmail, infrastructure-adapters-google-places, infrastructure-adapters-google-trends, infrastructure-adapters-shodan, infrastructure-adapters-yelp, infrastructure-agent, … (82 total).
_What this crate deliberately does NOT own is a judgment. No committed registry records one for it, so none is stated here._
Where it sits
| Tier (ontology) | infrastructure |
| Architectural role (taxonomy) | unclassified (baselined) |
| Location | crates/infrastructure/web-asset-discoverer |
| Vocabulary in force (lexicon) | current |
Dependencies
Runtime, from outside the workspace.
| Crate | Requirement | Features | Optional | Only on |
|---|---|---|---|---|
scraper | ^0.20 | — | no | always |
url | ^2 | — | no | always |
Development. None.
Build. None.
Depended on by. Nothing in this workspace.
Feature flags
No Cargo features are defined: every capability is unconditional, so no consumer can receive a half-wired crate.
Targets
| Kind | Name | Source |
|---|---|---|
| lib | infrastructure_web_asset_discoverer | `src/lib.rs` |
Error model
No public error type was detected: no public item declares a type named *Error, and no public signature returns one.
Operational characteristics
| Property | Evidence |
|---|---|
| async public surface | none detected |
| async runtime | none detected |
| database access | none detected |
| network I/O | none detected |
| unsafe code | none detected |
| environment variables | none detected |
No unsafe block, unsafe fn, unsafe impl or unsafe trait was found by the parser anywhere in this crate's source.
Configuration
No environment variable is read with a literal name anywhere in this crate. A variable whose key is computed at run time cannot be listed here, and is not claimed to be absent.
Related capabilities
No workspace crate depends on this one.
Verification
| Kind | Count |
|---|---|
| Unit tests | 15 |
| Integration tests | 0 |
| Examples | 0 |
| Doctests | 0 |
Evidence by module. How often each public module is named by something executable.
| Module | Tests | Examples | Consumers |
|---|---|---|---|
discoverer | 4 | 0 | 0 |
What the tests establish, by name:
test_asset_kind_display_matches_source_asset_type_strings—src/discoverer.rstest_cap_at_does_not_truncate_input_under_the_limit—src/discoverer.rstest_cap_at_snaps_to_char_boundary_and_never_panics—src/discoverer.rstest_cap_at_truncates_oversized_input_and_reports_it—src/discoverer.rstest_deduplicate_assets—src/discoverer.rstest_discover_reports_input_capped_when_html_is_truncated—src/discoverer.rstest_extract_absolute_url—src/discoverer.rstest_extract_document—src/discoverer.rstest_extract_img_src—src/discoverer.rstest_extract_og_image—src/discoverer.rstest_extract_pdf—src/discoverer.rstest_multiple_asset_types—src/discoverer.rstest_skip_data_urls—src/discoverer.rstest_skip_javascript_url—src/discoverer.rstest_srcset_extraction—src/discoverer.rs
Documentation coverage
| Measure | Documented | Total |
|---|---|---|
| Public items with rustdoc | 6 | 6 |
Public modules with a //! block | 1 | 1 |
pie showData
title Public items with rustdoc
"Documented" : 6
"No rustdoc detected" : 0
Metrics
| Metric | Value |
|---|---|
| Rust source files | 2 |
| Source lines | 567 |
| Code lines | 393 |
| Public API items | 6 |
| Public modules | 1 |
| Tests | 15 |
| Examples | 0 |
| Cargo features | 0 |
| Direct runtime dependencies | 2 |
| Workspace reverse dependencies | 0 |
pie showData
title Public API by kind
"enum" : 1
"method" : 2
"struct" : 3
pie showData
title Rust source composition
"Code" : 393
"Blank or comment" : 174
Generation
Rendered by tools-corpus corpus readme from repository evidence alone, renderer schema 2, lexicon current. No model, network service or database was consulted. Regenerate with tools-corpus corpus readme --write; verify with --check.