infrastructure tier

`infrastructure-web-wayback-archive`

Wayback Machine / CDX API client implementing foundation-web-analysis::Analyzer -- historical snapshot summary (site age, redesign detection, change-frequency estimate) from web.archive.org's CDX index

Wayback Machine / CDX API client implementing foundation-web-analysis::Analyzer -- historical snapshot summary (site age, redesign detection, change-frequency estimate) from web.archive.org's CDX index

Tier	infrastructure
Role	unclassified (baselined)
Path	`crates/infrastructure/web-wayback-archive`
Edition	2021
Targets	infrastructure_web_wayback_archive
Public items	27 across 2 modules
Tests	39

What it is for

# infrastructure-web-wayback-archive

Wayback Machine / CDX API client analyzer implementing foundation_web_analysis::Analyzer. Queries web.archive.org's CDX index for every known historical snapshot of a URL and derives a summary: site age, activity by year, estimated redesign dates, content-change frequency, and the set of distinct historical URLs observed.

Lifted from plumber/src/analyzers/wayback.rs and plumber/src/models/historical_data.rs (sprint 3.46, wave 3 -- see CHANGELOG.md for provenance).

Capabilities

`analyzer (other)`

Wayback Machine / CDX API historical-snapshot analyzer.

Item
`pub const DEFAULT_MAX_BYTES: usize`
`pub const DEFAULT_CDX_DELAY: Duration`

`WaybackArchiveAnalyzer`

Wayback Machine / CDX API historical-snapshot analyzer.

Item
`pub struct WaybackArchiveAnalyzer`
`WaybackArchiveAnalyzer :: fn new() -> std::result::Result <Self, infrastructure_fetcher::SafeHttpError>`
`WaybackArchiveAnalyzer :: fn with_max_bytes(mut self, max_bytes : usize) -> Self`
`WaybackArchiveAnalyzer :: fn with_cdx_delay(mut self, delay : Duration) -> Self`
`WaybackArchiveAnalyzer :: fn default() -> Self`
`WaybackArchiveAnalyzer :: fn name(& self) -> & 'static str`
`WaybackArchiveAnalyzer :: async fn analyze(& self, organization_id : Uuid, website_url : & str) -> Result <AnalysisResult>`
`WaybackArchiveAnalyzer :: fn cache_ttl_days(& self) -> u32`

`CdxRecord`

Historical-snapshot data model.

Item
`pub struct CdxRecord`
`CdxRecord :: fn from_json_array(arr : & serde_json::Value) -> Option <Self>`
`CdxRecord :: fn parsed_timestamp(& self) -> Option <DateTime <Utc>>`
`CdxRecord :: fn to_snapshot(& self, organization_id : Uuid) -> Option <WaybackSnapshot>`

`ChangeFrequency`

Historical-snapshot data model.

Item
`pub enum ChangeFrequency`
`ChangeFrequency :: fn fmt(& self, f : & mut std::fmt::Formatter <'_>) -> std::fmt::Result`

`HistoricalSummary`

Historical-snapshot data model.

Item
`pub struct HistoricalSummary`
`HistoricalSummary :: fn default() -> Self`
`HistoricalSummary :: fn new(organization_id : Uuid, domain : & str) -> Self`
`HistoricalSummary :: fn site_age_years(& self) -> Option <f32>`
`HistoricalSummary :: fn is_established(& self) -> bool`
`HistoricalSummary :: fn recently_redesigned(& self) -> bool`

`TechHistoryEntry`

Historical-snapshot data model.

Item
`pub struct TechHistoryEntry`

`WaybackSnapshot`

Historical-snapshot data model.

Item
`pub struct WaybackSnapshot`
`WaybackSnapshot :: fn default() -> Self`
`WaybackSnapshot :: fn new(organization_id : Uuid, original_url : & str, archive_url : & str, timestamp : DateTime <Utc>,) -> Self`
`WaybackSnapshot :: fn wayback_url(& self) -> String`

How to use it

No examples/ target and no doctest in this crate's rustdoc. The tests listed under Verification are the closest executable usage.

Module structure

infrastructure_web_wayback_archive

analyzer
models

flowchart TD
  n_infrastructure_web_wayback_archive["infrastructure_web_wayback_archive"]
  n_infrastructure_web_wayback_archive --> n_analyzer["analyzer"]
  n_infrastructure_web_wayback_archive --> n_models["models"]

Public surface

`analyzer`

Item	What it is
`pub const DEFAULT_MAX_BYTES: usize`	Response body byte cap for a CDX API query (Gate 1.5.6 input limit)
`pub const DEFAULT_CDX_DELAY: Duration`	Courtesy delay between successive CDX API calls -- be respectful to the Internet Archive's free public index, matching the source's own `WAYBACK_DELAY`.
`pub struct WaybackArchiveAnalyzer`	Analyzer for a website's historical presence in the Wayback Machine, via the CDX API (`web.archive.org/cdx/search/cdx`)
`WaybackArchiveAnalyzer :: fn new() -> std::result::Result <Self, infrastructure_fetcher::SafeHttpError>`	Build an analyzer with the default byte cap (`DEFAULT_MAX_BYTES`), timeout (`infrastructure_fetcher::safe_http::DEFAULT_TIMEOUT`, 30s), and CDX courtesy delay (`DEFAULT_CDX_DELAY`, 500ms).
`WaybackArchiveAnalyzer :: fn with_max_bytes(mut self, max_bytes : usize) -> Self`	Override the response body byte cap.
`WaybackArchiveAnalyzer :: fn with_cdx_delay(mut self, delay : Duration) -> Self`	Override the courtesy delay between CDX API calls (test-only escape hatch so a two-call `analyze()` test isn't slowed by the production default -- matches `DomainIntelAnalyzer::with_geo_delay`).
`WaybackArchiveAnalyzer :: fn default() -> Self`	# Panics Panics if the underlying `SafeHttpClient` cannot be built (TLS backend init failure) -- matches the source's `expect`-on-construction posture (`Client::builder()...expect("Failed to build HTTP client")`) and `TechDetector`/`AdaChecker`'s same deviation.
`WaybackArchiveAnalyzer :: fn name(& self) -> & 'static str`	—
`WaybackArchiveAnalyzer :: async fn analyze(& self, organization_id : Uuid, website_url : & str) -> Result <AnalysisResult>`	—
`WaybackArchiveAnalyzer :: fn cache_ttl_days(& self) -> u32`	—

`models`

Item	What it is
`pub struct WaybackSnapshot`	A single Wayback Machine snapshot of a URL.
`WaybackSnapshot :: fn default() -> Self`	—
`WaybackSnapshot :: fn new(organization_id : Uuid, original_url : & str, archive_url : & str, timestamp : DateTime <Utc>,) -> Self`	—
`WaybackSnapshot :: fn wayback_url(& self) -> String`	Format as a `web.archive.org` playback URL for this snapshot.
`pub struct HistoricalSummary`	Summary of a website's historical presence in the Wayback Machine.
`pub enum ChangeFrequency`	How often a site's content appears to change, estimated from unique CDX digests over the observed time span.
`ChangeFrequency :: fn fmt(& self, f : & mut std::fmt::Formatter <'_>) -> std::fmt::Result`	—
`pub struct TechHistoryEntry`	One historical technology-detection entry (CMS/framework as observed at a point in time)
`HistoricalSummary :: fn default() -> Self`	—
`HistoricalSummary :: fn new(organization_id : Uuid, domain : & str) -> Self`	—
`HistoricalSummary :: fn site_age_years(& self) -> Option <f32>`	Site age in years, derived from `site_age_days`.
`HistoricalSummary :: fn is_established(& self) -> bool`	`true` when the site has more than 3 years of Wayback history.
`HistoricalSummary :: fn recently_redesigned(& self) -> bool`	`true` when the last detected redesign was within the last year.
`pub struct CdxRecord`	One row of the Wayback Machine CDX API's `output=json` response, in the field order requested by `fl=urlkey,timestamp,original,mimetype, statuscode,digest,length`.
`CdxRecord :: fn from_json_array(arr : & serde_json::Value) -> Option <Self>`	Parse one row from the CDX API's JSON-array-of-arrays response shape
`CdxRecord :: fn parsed_timestamp(& self) -> Option <DateTime <Utc>>`	Parse the CDX `timestamp` field (`YYYYMMDDHHMMSS`, UTC) into a `DateTime<Utc>`.
`CdxRecord :: fn to_snapshot(& self, organization_id : Uuid) -> Option <WaybackSnapshot>`	Convert to a `WaybackSnapshot`

Re-exports. Exported here, defined elsewhere.

Export	Defined in
`{CdxRecord,ChangeFrequency,HistoricalSummary,TechHistoryEntry,WaybackSnapshot,}`	`models::{CdxRecord,ChangeFrequency,HistoricalSummary,TechHistoryEntry,WaybackSnapshot,}`
`{WaybackArchiveAnalyzer,DEFAULT_CDX_DELAY,DEFAULT_MAX_BYTES}`	`analyzer::{WaybackArchiveAnalyzer,DEFAULT_CDX_DELAY,DEFAULT_MAX_BYTES}`

Boundary

Reaches into foundation.

Shares tier infrastructure with 82 other crates: infrastructure-acquire, infrastructure-adapters-google-calendar, infrastructure-adapters-google-gmail, infrastructure-adapters-google-places, infrastructure-adapters-google-trends, infrastructure-adapters-shodan, infrastructure-adapters-yelp, infrastructure-agent, … (82 total).

_What this crate deliberately does NOT own is a judgment. No committed registry records one for it, so none is stated here._

Where it sits

Tier (ontology)	`infrastructure`
Architectural role (taxonomy)	`unclassified (baselined)`
Location	`crates/infrastructure/web-wayback-archive`
Vocabulary in force (lexicon)	`current`

Tier flow. Which tiers this crate's own edges cross.

flowchart LR
  n_infrastructure["infrastructure"] --> n_foundation["foundation"]

Dependencies

Runtime, in this workspace.

Crate	Tier	Optional	Only on
`foundation-web-analysis`	foundation	no	always
`infrastructure-fetcher`	infrastructure	no	always

Runtime, from outside the workspace.

Crate	Requirement	Features	Optional	Only on
`async-trait`	`^0.1`	—	no	always
`chrono`	`^0.4`	`serde`	no	always
`serde`	`^1`	`derive`	no	always
`serde_json`	`^1`	—	no	always
`thiserror`	`^2`	—	no	always
`tokio`	`^1`	`full`	no	always
`url`	`^2`	—	no	always
`uuid`	`^1`	`v4`, `v7`, `serde`, `js`	no	always

Development, from outside the workspace.

Crate	Requirement	Features	Optional	Only on
`tokio`	`^1`	`full`, `full`, `test-util`	no	always
`tokio-test`	`^0.4`	—	no	always

Build. None.

Depended on by. Nothing in this workspace.

Signal flow — what reaches this crate, and what it reaches.

flowchart LR
  SELF["infrastructure-web-wayback-archive"]
  SELF -->|runtime| n_foundation_web_analysis["foundation-web-analysis"]
  SELF -->|runtime| n_infrastructure_fetcher["infrastructure-fetcher"]
  classDef self fill:#1f883d,stroke:#1f883d,color:#fff;
  class SELF self;

Feature flags

No Cargo features are defined: every capability is unconditional, so no consumer can receive a half-wired crate.

Targets

Kind	Name	Source
lib	`infrastructure_web_wayback_archive`	`src/lib.rs`

Error model

No public error type was detected: no public item declares a type named *Error, and no public signature returns one.

Operational characteristics

Property	Evidence
async public surface	yes
async runtime	yes
database access	none detected
network I/O	none detected
unsafe code	none detected
environment variables	none detected

No unsafe block, unsafe fn, unsafe impl or unsafe trait was found by the parser anywhere in this crate's source.

Configuration

No environment variable is read with a literal name anywhere in this crate. A variable whose key is computed at run time cannot be listed here, and is not claimed to be absent.

No workspace crate depends on this one.

Verification

Kind	Count
Unit tests	39
Integration tests	0
Examples	0
Doctests	0

Evidence by module. How often each public module is named by something executable.

Module	Tests	Examples	Consumers
`analyzer`	3	0	0
`models`	5	0	0

What the tests establish, by name:

analyze_returns_structured_failure_for_invalid_url — src/analyzer.rs
analyze_snapshots_collects_unique_historical_urls — src/analyzer.rs
analyze_snapshots_computes_years_active_and_average — src/analyzer.rs
analyze_snapshots_empty_records_yields_zero_total — src/analyzer.rs
analyze_snapshots_sets_total_and_first_archive_date — src/analyzer.rs
analyze_snapshots_sorts_by_timestamp — src/analyzer.rs
cache_ttl_is_thirty_days — src/analyzer.rs
detect_redesigns_collapses_changes_within_six_months — src/analyzer.rs
detect_redesigns_empty_when_digest_never_changes — src/analyzer.rs
detect_redesigns_flags_digest_change_across_months — src/analyzer.rs
estimate_change_frequency_daily_for_frequent_unique_digests — src/analyzer.rs
estimate_change_frequency_rarely_for_one_change_over_years — src/analyzer.rs
estimate_change_frequency_unknown_for_fewer_than_two_snapshots — src/analyzer.rs
estimate_change_frequency_unknown_for_zero_snapshots — src/analyzer.rs
name_is_wayback_archive_analyzer — src/analyzer.rs
requires_link_graph_defaults_to_false — src/analyzer.rs
cdx_record_from_json_array_parses_full_row — src/models.rs
cdx_record_from_json_array_rejects_non_string_field — src/models.rs
cdx_record_from_json_array_rejects_short_row — src/models.rs
cdx_record_parsed_timestamp_none_for_malformed — src/models.rs
cdx_record_parsed_timestamp_parses_wayback_format — src/models.rs
cdx_record_to_snapshot_converts_fields — src/models.rs
cdx_record_to_snapshot_handles_non_numeric_status_and_length — src/models.rs
cdx_record_to_snapshot_none_for_unparseable_timestamp — src/models.rs
change_frequency_display_matches_source_strings — src/models.rs
historical_summary_new_sets_org_and_domain — src/models.rs
historical_summary_serialization_round_trips — src/models.rs
is_established_false_under_three_years — src/models.rs
is_established_false_when_age_unknown — src/models.rs
is_established_true_over_three_years — src/models.rs
_… 9 more_

Documentation coverage

Measure	Documented	Total
Public items with rustdoc	19	27
Public modules with a `//!` block	2	2

pie showData
    title Public items with rustdoc
    "Documented" : 19
    "No rustdoc detected" : 8

Metrics

Metric	Value
Rust source files	3
Source lines	1099
Code lines	860
Public API items	27
Public modules	2
Tests	39
Examples	0
Cargo features	0
Direct runtime dependencies	10
Workspace reverse dependencies	0

pie showData
    title Public API by kind
    "constant" : 2
    "enum" : 1
    "method" : 19
    "struct" : 5

pie showData
    title Rust source composition
    "Code" : 860
    "Blank or comment" : 239

Generation

Rendered by tools-corpus corpus readme from repository evidence alone, renderer schema 2, lexicon current. No model, network service or database was consulted. Regenerate with tools-corpus corpus readme --write; verify with --check.

All infrastructure · Manual