infrastructure-web-backlink-finder

Backlink discovery/classification implementing foundation-web-analysis::Analyzer -- Bing/DuckDuckGo search-result parsing, directory/review/social/forum/editorial link-type classification, and a per-domain backlink summary

Tier	infrastructure
Role	unclassified (baselined)
Path	`crates/infrastructure/web-backlink-finder`
Edition	2021
Targets	infrastructure_web_backlink_finder
Public items	29 across 2 modules
Tests	43

What it is for

# infrastructure-web-backlink-finder

Backlink discovery/classification implementing foundation_web_analysis::Analyzer. Searches Bing and DuckDuckGo for pages linking to a target domain, classifies each discovered link (directory / review / social / forum / editorial / unknown) by its source domain, and produces a per-domain models::BacklinkSummary.

Lifted from plumber/src/analyzers/backlink_finder.rs and plumber/src/models/backlinks.rs (sprint 3.46, wave 2 -- see CHANGELOG.md for provenance).

Capabilities

`finder (other)`

Backlink discovery/classification analyzer.

Item
`pub const DEFAULT_MAX_BYTES: usize`

`BacklinkFinder`

Backlink discovery/classification analyzer.

Item
`pub struct BacklinkFinder`
`BacklinkFinder :: fn new() -> std::result::Result <Self, infrastructure_fetcher::SafeHttpError>`
`BacklinkFinder :: fn with_max_bytes(mut self, max_bytes : usize) -> Self`
`BacklinkFinder :: fn with_delay_range(mut self, min_secs : u64, max_secs : u64) -> Self`
`BacklinkFinder :: fn with_search_bases(mut self, bing_base : & str, duckduckgo_base : & str) -> Self`
`BacklinkFinder :: fn default() -> Self`
`BacklinkFinder :: fn name(& self) -> & 'static str`
`BacklinkFinder :: async fn analyze(& self, organization_id : Uuid, website_url : & str) -> Result <AnalysisResult>`
`BacklinkFinder :: fn requires_link_graph(& self) -> bool`
`BacklinkFinder :: fn cache_ttl_days(& self) -> u32`

`Backlink`

Backlink data model.

Item
`pub struct Backlink`
`Backlink :: fn default() -> Self`

`BacklinkSummary`

Backlink data model.

Item
`pub struct BacklinkSummary`
`BacklinkSummary :: fn new(organization_id : Uuid) -> Self`
`BacklinkSummary :: fn dofollow_percentage(& self) -> f32`
`BacklinkSummary :: fn avg_links_per_domain(& self) -> f32`

`DirectorySites`

Backlink data model.

Item
`pub struct DirectorySites`
`DirectorySites :: fn is_directory(domain : & str) -> bool`
`DirectorySites :: fn known_directories() -> Vec <& 'static str>`
`DirectorySites :: fn is_review_site(domain : & str) -> bool`
`DirectorySites :: fn review_sites() -> Vec <& 'static str>`
`DirectorySites :: fn is_social_media(domain : & str) -> bool`

`DiscoverySource`

Backlink data model.

Item
`pub enum DiscoverySource`
`DiscoverySource :: fn fmt(& self, f : & mut std::fmt::Formatter <'_>) -> std::fmt::Result`

`DomainLinkCount`

Backlink data model.

Item
`pub struct DomainLinkCount`

`LinkType`

Backlink data model.

Item
`pub enum LinkType`
`LinkType :: fn fmt(& self, f : & mut std::fmt::Formatter <'_>) -> std::fmt::Result`
`LinkType :: fn from_str(s : & str) -> Result <Self, Self::Err>`

How to use it

No examples/ target and no doctest in this crate's rustdoc. The tests listed under Verification are the closest executable usage.

Module structure

infrastructure_web_backlink_finder

finder
models

flowchart TD
  n_infrastructure_web_backlink_finder["infrastructure_web_backlink_finder"]
  n_infrastructure_web_backlink_finder --> n_finder["finder"]
  n_infrastructure_web_backlink_finder --> n_models["models"]

Public surface

`finder`

Item	What it is
`pub const DEFAULT_MAX_BYTES: usize`	Response body byte cap for a fetched search-results page (Gate 1.5.6 input limit).
`pub struct BacklinkFinder`	Analyzer for finding backlinks via Bing and DuckDuckGo `link:` searches
`BacklinkFinder :: fn new() -> std::result::Result <Self, infrastructure_fetcher::SafeHttpError>`	Build a finder with the default byte cap (`DEFAULT_MAX_BYTES`), timeout (`infrastructure_fetcher::safe_http::DEFAULT_TIMEOUT`, 30s), and inter-search delay range (30-60s, matching the source's own `min_delay_secs`/`max_delay_secs` defaults).
`BacklinkFinder :: fn with_max_bytes(mut self, max_bytes : usize) -> Self`	Override the response body byte cap.
`BacklinkFinder :: fn with_delay_range(mut self, min_secs : u64, max_secs : u64) -> Self`	Override the inter-search delay range, in seconds.
`BacklinkFinder :: fn with_search_bases(mut self, bing_base : & str, duckduckgo_base : & str) -> Self`	Override the search-endpoint base URLs
`BacklinkFinder :: fn default() -> Self`	# Panics Panics if the underlying `SafeHttpClient` cannot be built (TLS backend init failure) -- matches the source's `expect`-on-construction posture (`Client::builder()...expect("Failed to build HTTP client")`).
`BacklinkFinder :: fn name(& self) -> & 'static str`	—
`BacklinkFinder :: async fn analyze(& self, organization_id : Uuid, website_url : & str) -> Result <AnalysisResult>`	—
`BacklinkFinder :: fn requires_link_graph(& self) -> bool`	—
`BacklinkFinder :: fn cache_ttl_days(& self) -> u32`	—

`models`

Item	What it is
`pub struct Backlink`	A single backlink discovered for an organization's website
`Backlink :: fn default() -> Self`	—
`pub enum LinkType`	Classification of the linking relationship a `Backlink` represents.
`LinkType :: fn fmt(& self, f : & mut std::fmt::Formatter <'_>) -> std::fmt::Result`	—
`LinkType :: fn from_str(s : & str) -> Result <Self, Self::Err>`	—
`pub enum DiscoverySource`	Where a `Backlink` was discovered.
`DiscoverySource :: fn fmt(& self, f : & mut std::fmt::Formatter <'_>) -> std::fmt::Result`	—
`pub struct BacklinkSummary`	Aggregate backlink-profile summary for one organization's website.
`pub struct DomainLinkCount`	—
`BacklinkSummary :: fn new(organization_id : Uuid) -> Self`	—
`BacklinkSummary :: fn dofollow_percentage(& self) -> f32`	Percentage (0.0-100.0) of backlinks that are dofollow
`BacklinkSummary :: fn avg_links_per_domain(& self) -> f32`	Average backlinks per unique linking domain
`pub struct DirectorySites`	Known business-directory / review / social-media domains used to classify a backlink's `LinkType` by its source domain.
`DirectorySites :: fn is_directory(domain : & str) -> bool`	—
`DirectorySites :: fn known_directories() -> Vec <& 'static str>`	—
`DirectorySites :: fn is_review_site(domain : & str) -> bool`	—
`DirectorySites :: fn review_sites() -> Vec <& 'static str>`	—
`DirectorySites :: fn is_social_media(domain : & str) -> bool`	—

Re-exports. Exported here, defined elsewhere.

Export	Defined in
`{Backlink,BacklinkSummary,DirectorySites,DiscoverySource,DomainLinkCount,LinkType,}`	`models::{Backlink,BacklinkSummary,DirectorySites,DiscoverySource,DomainLinkCount,LinkType,}`
`{BacklinkFinder,DEFAULT_MAX_BYTES}`	`finder::{BacklinkFinder,DEFAULT_MAX_BYTES}`

Boundary

Reaches into foundation.

Shares tier infrastructure with 82 other crates: infrastructure-acquire, infrastructure-adapters-google-calendar, infrastructure-adapters-google-gmail, infrastructure-adapters-google-places, infrastructure-adapters-google-trends, infrastructure-adapters-shodan, infrastructure-adapters-yelp, infrastructure-agent, … (82 total).

_What this crate deliberately does NOT own is a judgment. No committed registry records one for it, so none is stated here._

Where it sits

Tier (ontology)	`infrastructure`
Architectural role (taxonomy)	`unclassified (baselined)`
Location	`crates/infrastructure/web-backlink-finder`
Vocabulary in force (lexicon)	`current`

Tier flow. Which tiers this crate's own edges cross.

flowchart LR
  n_infrastructure["infrastructure"] --> n_foundation["foundation"]

Dependencies

Runtime, in this workspace.

Crate	Tier	Optional	Only on
`foundation-web-analysis`	foundation	no	always
`infrastructure-fetcher`	infrastructure	no	always

Runtime, from outside the workspace.

Crate	Requirement	Features	Optional	Only on
`async-trait`	`^0.1`	—	no	always
`chrono`	`^0.4`	`serde`	no	always
`rand`	`^0.8`	—	no	always
`scraper`	`^0.20`	—	no	always
`serde`	`^1`	`derive`	no	always
`serde_json`	`^1`	—	no	always
`thiserror`	`^2`	—	no	always
`tokio`	`^1`	`full`	no	always
`tracing`	`^0.1`	—	no	always
`url`	`^2`	—	no	always
`urlencoding`	`^2`	—	no	always
`uuid`	`^1`	`v4`, `v7`, `serde`, `js`	no	always

Development, from outside the workspace.

Crate	Requirement	Features	Optional	Only on
`tokio`	`^1`	`full`, `full`, `test-util`	no	always
`tokio-test`	`^0.4`	—	no	always

Build. None.

Depended on by. Nothing in this workspace.

Signal flow — what reaches this crate, and what it reaches.

flowchart LR
  SELF["infrastructure-web-backlink-finder"]
  SELF -->|runtime| n_foundation_web_analysis["foundation-web-analysis"]
  SELF -->|runtime| n_infrastructure_fetcher["infrastructure-fetcher"]
  classDef self fill:#1f883d,stroke:#1f883d,color:#fff;
  class SELF self;

Feature flags

No Cargo features are defined: every capability is unconditional, so no consumer can receive a half-wired crate.

Targets

Kind	Name	Source
lib	`infrastructure_web_backlink_finder`	`src/lib.rs`

Error model

No public error type was detected: no public item declares a type named *Error, and no public signature returns one.

Operational characteristics

Property	Evidence
async public surface	yes
async runtime	yes
database access	none detected
network I/O	none detected
unsafe code	none detected
environment variables	none detected

No unsafe block, unsafe fn, unsafe impl or unsafe trait was found by the parser anywhere in this crate's source.

Configuration

No environment variable is read with a literal name anywhere in this crate. A variable whose key is computed at run time cannot be listed here, and is not claimed to be absent.

No workspace crate depends on this one.

Verification

Kind	Count
Unit tests	43
Integration tests	0
Examples	0
Doctests	0

Evidence by module. How often each public module is named by something executable.

Module	Tests	Examples	Consumers
`finder`	2	0	0
`models`	6	0	0

What the tests establish, by name:

analyze_fails_on_invalid_url — src/finder.rs
analyze_records_warnings_not_errors_when_both_searches_fail — src/finder.rs
cache_ttl_is_seven_days — src/finder.rs
classify_link_blog_keyword_in_domain — src/finder.rs
classify_link_directory_wins_over_review_when_domain_is_in_both_lists — src/finder.rs
classify_link_forum_keyword_in_domain — src/finder.rs
classify_link_review_site_not_also_a_directory — src/finder.rs
classify_link_social_media_domain_not_in_directory_or_review_lists — src/finder.rs
classify_link_unrecognized_domain_is_unknown — src/finder.rs
create_summary_classifies_anchor_text — src/finder.rs
create_summary_counts_by_link_type — src/finder.rs
create_summary_counts_totals_and_dofollow_nofollow — src/finder.rs
create_summary_top_domains_sorted_descending_by_count — src/finder.rs
decode_duckduckgo_redirect_decodes_uddg_wrapped_url — src/finder.rs
decode_duckduckgo_redirect_passes_through_a_direct_url — src/finder.rs
extract_domain_parses_host_from_valid_url — src/finder.rs
extract_domain_returns_none_for_invalid_url — src/finder.rs
manual_review_is_none_when_no_backlinks_found — src/finder.rs
manual_review_present_and_describes_counts_when_backlinks_found — src/finder.rs
name_is_backlink_finder — src/finder.rs
parse_bing_results_extracts_url_title_and_snippet — src/finder.rs
parse_bing_results_returns_empty_vec_when_no_result_rows — src/finder.rs
parse_duckduckgo_results_decodes_redirect_wrapped_url — src/finder.rs
parse_duckduckgo_results_extracts_direct_url — src/finder.rs
parse_duckduckgo_results_returns_empty_vec_when_no_result_rows — src/finder.rs
requires_link_graph_is_false — src/finder.rs
results_to_backlinks_classifies_and_marks_dofollow — src/finder.rs
results_to_backlinks_skips_duplicate_urls_already_seen — src/finder.rs
results_to_backlinks_skips_self_links — src/finder.rs
avg_links_per_domain_computed_from_counts — src/models.rs
_… 13 more_

Documentation coverage

Measure	Documented	Total
Public items with rustdoc	14	29
Public modules with a `//!` block	2	2

pie showData
    title Public items with rustdoc
    "Documented" : 14
    "No rustdoc detected" : 15

Metrics

Metric	Value
Rust source files	3
Source lines	1330
Code lines	1034
Public API items	29
Public modules	2
Tests	43
Examples	0
Cargo features	0
Direct runtime dependencies	14
Workspace reverse dependencies	0

pie showData
    title Public API by kind
    "constant" : 1
    "enum" : 2
    "method" : 21
    "struct" : 5

pie showData
    title Rust source composition
    "Code" : 1034
    "Blank or comment" : 296

Generation

Rendered by tools-corpus corpus readme from repository evidence alone, renderer schema 2, lexicon current. No model, network service or database was consulted. Regenerate with tools-corpus corpus readme --write; verify with --check.

What it is for

Capabilities

finder (other)

BacklinkFinder

Backlink

BacklinkSummary

DirectorySites

DiscoverySource

DomainLinkCount

LinkType

How to use it

Module structure

Public surface

Boundary

Where it sits

Dependencies

Feature flags

Targets

Error model

Operational characteristics

Configuration

Related capabilities

Verification

Documentation coverage

Metrics

Generation

`finder (other)`

`BacklinkFinder`

`Backlink`

`BacklinkSummary`

`DirectorySites`

`DiscoverySource`

`DomainLinkCount`

`LinkType`