The E2E populate->verify->report runner engine (sprint 0.79 P2): a Check trait, a Driver abstraction over infrastructure-browser-automation, per-run database isolation from a pre-migrated TEMPLATE, reporting over operations-test-results, and a tiered + bounded-parallel runner. The registry-driven GENERATOR and FieldKind test adapters are P3 and do NOT live here.
| Tier | operations |
| Role | unclassified (baselined) |
| Path | crates/operations/e2e-harness |
| Edition | 2021 |
| Targets | consumer_suite, operations_e2e_harness, fake_driver, generator_tests, kit_checks_tests, layer_tests, live_browser_tests, run_db_tests, runner_tests |
| Public items | 237 across 27 modules |
| Tests | 143 |
What it is for
# operations-e2e-harness
The populate→verify→report runner engine for end-to-end checks — sprint 0.79 Phase 2.
This crate is the engine: a Check interface, a Driver abstraction over infrastructure-browser-automation, per-run database isolation (RunDb) cloned from a pre-migrated template, reporting over operations-test-results (report), and a tiered, bounded-parallel Runner. Hand-written checks run on this engine today; the registry-driven generator and the FieldKind test adapters are P3. The proactive assertion layers (layers — a11y / visual / i18n / viewport / state / perf / flaky / console+network / the PII-on-public-surface security guard) are P4: each is a composable assertion folded into ANY check's outcome, producing findings (and perf metric samples) into the same run report. The reusable kit checks (kit_checks — the CMS template-gallery / composed-page / composer / media-picker / media-authz-at-UI coverage, generated where registry-shaped and hand-written where bespoke) are P5, consumable by ANY kit consumer; the copy-this consumer suite lives in examples/consumer_suite.rs.
The shape of a run
1. Provision isolation — clone the template DB into a per-run database (RunDb::create_from_template); point the per-run server at the clone. 2. Build a RunCtx (driver + base URL + per-run DB pool + run id), reset the browser session so no stale cache/cookies leak in. 3. Runner::run schedules the Tier-selected checks (optionally in bounded parallel), running each check's idempotent populate then read-only verify. 4. Every check's outcome + findings fold into ONE run_id recorded to operations-test-results. 5. Drop the per-run DB (RunDb::drop_db); Drop is a best-effort backstop.
The three footguns this engine is built against
- Oracle drift — the contract types (test-ids, manifest, role policy)
live in the thin operations-e2e-contract crate (P1), the single Rust source of truth this engine keys off; nothing is mirrored.
- No-op theater — a check's
verifymust prove something (a marker
rendered, no console errors, DB state); a passing CheckOutcome with no assertion is the author's responsibility, and the FieldKind adapters that enforce model-change+save+reload+SSR are layered on in P3.
- State-pollution + stale-cache — per-run DB from a template
(run_db) plus a browser-session reset (Driver::reset_session) aligned to the run lifecycle, so the DB is never reset under a live browser.
Capabilities
adapter (other)
FieldKind test adapters — item 1 and 3 of the locked architecture.
| Item |
|---|
fn selector_for(id : & TestId) -> String |
fn known_value_for(id : & TestId) -> String |
FieldAdapter
FieldKind test adapters — item 1 and 3 of the locked architecture.
| Item |
|---|
pub trait FieldAdapter |
fn adapter_for(control : ControlKind) -> Option <Box <dyn FieldAdapter>> |
MediaPickerAdapter
FieldKind test adapters — item 1 and 3 of the locked architecture.
| Item |
|---|
pub struct MediaPickerAdapter |
MediaPickerAdapter :: fn control(& self) -> ControlKind |
MediaPickerAdapter :: async fn drive(& self, driver : & dyn Driver, save_reload : & dyn SaveReload, id : & TestId, value : & str,) -> Result <MutationProof> |
SaveReload
FieldKind test adapters — item 1 and 3 of the locked architecture.
| Item |
|---|
pub trait SaveReload |
async fn drive_field(driver : & dyn Driver, save_reload : & dyn SaveReload, id : & TestId, value : & str,) -> Result <Option <MutationProof>> |
SelectAdapter
FieldKind test adapters — item 1 and 3 of the locked architecture.
| Item |
|---|
pub struct SelectAdapter |
SelectAdapter :: fn control(& self) -> ControlKind |
SelectAdapter :: async fn drive(& self, driver : & dyn Driver, save_reload : & dyn SaveReload, id : & TestId, value : & str,) -> Result <MutationProof> |
TextInputAdapter
FieldKind test adapters — item 1 and 3 of the locked architecture.
| Item |
|---|
pub struct TextInputAdapter |
TextInputAdapter :: fn control(& self) -> ControlKind |
TextInputAdapter :: async fn drive(& self, driver : & dyn Driver, save_reload : & dyn SaveReload, id : & TestId, value : & str,) -> Result <MutationProof> |
TextareaAdapter
FieldKind test adapters — item 1 and 3 of the locked architecture.
| Item |
|---|
pub struct TextareaAdapter |
TextareaAdapter :: fn control(& self) -> ControlKind |
TextareaAdapter :: async fn drive(& self, driver : & dyn Driver, save_reload : & dyn SaveReload, id : & TestId, value : & str,) -> Result <MutationProof> |
Check
The Check interface: the populate→verify contract a consumer implements.
| Item |
|---|
pub trait Check |
CheckOutcome
The Check interface: the populate→verify contract a consumer implements.
| Item |
|---|
pub struct CheckOutcome |
CheckOutcome :: fn passed() -> Self |
CheckOutcome :: fn failed(detail : impl Into <String>) -> Self |
CheckOutcome :: fn with_finding(mut self, finding : Finding) -> Self |
CheckOutcome :: fn with_findings(mut self, findings : impl IntoIterator <Item = Finding>) -> Self |
CheckOutcome :: fn with_metric(mut self, metric : MetricInput) -> Self |
CheckOutcome :: fn with_metrics(mut self, metrics : impl IntoIterator <Item = MetricInput>) -> Self |
CheckOutcome :: fn is_pass(& self) -> bool |
CheckOutcome :: fn error(& self) -> Option <& str> |
CheckOutcome :: fn findings(& self) -> & Finding |
CheckOutcome :: fn metrics(& self) -> & MetricInput |
Tier
The Check interface: the populate→verify contract a consumer implements.
| Item |
|---|
pub enum Tier |
Tier :: fn runs_in(self, requested : Tier) -> bool |
completeness (other)
The registry-completeness gate — item 6 (US-0.79.6).
| Item |
|---|
fn assert_complete(source : & dyn DefinitionSource) |
CompletenessGap
The registry-completeness gate — item 6 (US-0.79.6).
| Item |
|---|
pub enum CompletenessGap |
CompletenessGap :: fn reason(& self) -> String |
fn completeness_report(source : & dyn DefinitionSource) -> Vec <CompletenessGap> |
Coverage
THE hard contract that kills no-op theater — item 4 (and footgun #2) of
| Item |
|---|
pub enum Coverage |
MissingProof
THE hard contract that kills no-op theater — item 4 (and footgun #2) of
| Item |
|---|
pub enum MissingProof |
MissingProof :: fn reason(self) -> & 'static str |
MutationProof
THE hard contract that kills no-op theater — item 4 (and footgun #2) of
| Item |
|---|
pub struct MutationProof |
MutationProof :: fn new(target : impl Into <String>, set_value : impl Into <String>) -> Self |
MutationProof :: fn target(& self) -> & str |
MutationProof :: fn set_value(& self) -> & str |
MutationProof :: fn dom_reflected(mut self, read_back : & str) -> Self |
MutationProof :: fn survived_reload(mut self, read_after_reload : & str) -> Self |
MutationProof :: fn rendered_surface(mut self, shown : bool) -> Self |
MutationProof :: fn ledger(& self) -> ProofLedger |
MutationProof :: fn into_coverage(self) -> Coverage |
MutationProof :: fn is_covered(& self) -> bool |
ProofLedger
THE hard contract that kills no-op theater — item 4 (and footgun #2) of
| Item |
|---|
pub struct ProofLedger |
crawl (other)
page renders the content that was AUTHORED, not just that its template renders.
| Item |
|---|
fn expected_rendered_values(metadata : & Value, policy : & CrawlPolicy) -> ExpectedContent |
fn missing_content(expected : & ExpectedContent, rendered_html : & str) -> Vec <MissingContent> |
fn rendered_contains(rendered_html : & str, value : & str) -> bool |
CrawlPolicy
page renders the content that was AUTHORED, not just that its template renders.
| Item |
|---|
pub struct CrawlPolicy |
CrawlPolicy :: fn default() -> Self |
ExpectedContent
page renders the content that was AUTHORED, not just that its template renders.
| Item |
|---|
pub struct ExpectedContent |
MissingContent
page renders the content that was AUTHORED, not just that its template renders.
| Item |
|---|
pub struct MissingContent |
PageRecord
page renders the content that was AUTHORED, not just that its template renders.
| Item |
|---|
pub struct PageRecord |
DefinitionSource
Discovery: the DefinitionSource seam + the DiscoveredDefinition /
| Item |
|---|
pub trait DefinitionSource |
DiscoveredDefinition
Discovery: the DefinitionSource seam + the DiscoveredDefinition /
| Item |
|---|
pub struct DiscoveredDefinition |
DiscoveredDefinition :: fn has_sample(& self) -> bool |
DiscoveredField
Discovery: the DefinitionSource seam + the DiscoveredDefinition /
| Item |
|---|
pub struct DiscoveredField |
DiscoveredField :: fn leaf(name : impl Into <String>, kind : FieldKind) -> Self |
DiscoveredField :: fn select(name : impl Into <String>, options : impl IntoIterator <Item = String>) -> Self |
DiscoveredField :: fn repeating(name : impl Into <String>, item_fields : impl IntoIterator <Item = DiscoveredField>,) -> Self |
DiscoveredField :: fn is_container(& self) -> bool |
CdpDriver:CdpDriver
The browser Driver abstraction and its live CDP implementation.
| Item |
|---|
pub struct CdpDriver |
CdpDriver:click
The browser Driver abstraction and its live CDP implementation.
| Item |
|---|
CdpDriver :: async fn click(& self, css_selector : & str) -> Result <()> |
CdpDriver:connect
The browser Driver abstraction and its live CDP implementation.
| Item |
|---|
CdpDriver :: fn connect(cdp_base_url : & str) -> Result <Self> |
CdpDriver:exists
The browser Driver abstraction and its live CDP implementation.
| Item |
|---|
CdpDriver :: async fn exists(& self, css_selector : & str) -> Result <bool> |
CdpDriver:navigate
The browser Driver abstraction and its live CDP implementation.
| Item |
|---|
CdpDriver :: async fn navigate(& self, url : & str) -> Result <()> |
CdpDriver:page
The browser Driver abstraction and its live CDP implementation.
| Item |
|---|
CdpDriver :: async fn page_html(& self) -> Result <String> |
CdpDriver:read
The browser Driver abstraction and its live CDP implementation.
| Item |
|---|
CdpDriver :: async fn read_text(& self, css_selector : & str) -> Result <String> |
CdpDriver :: async fn read_value(& self, css_selector : & str) -> Result <String> |
CdpDriver:reset
The browser Driver abstraction and its live CDP implementation.
| Item |
|---|
CdpDriver :: async fn reset_session(& self) -> Result <()> |
CdpDriver:run
The browser Driver abstraction and its live CDP implementation.
| Item |
|---|
CdpDriver :: async fn run_axe(& self) -> Result <Option <Value>> |
CdpDriver:take
The browser Driver abstraction and its live CDP implementation.
| Item |
|---|
CdpDriver :: async fn take_console(& self) -> Result <Vec <Value>> |
CdpDriver :: async fn take_network(& self) -> Result <Vec <Value>> |
CdpDriver:type
The browser Driver abstraction and its live CDP implementation.
| Item |
|---|
CdpDriver :: async fn type_text(& self, css_selector : & str, text : & str) -> Result <()> |
Driver
The browser Driver abstraction and its live CDP implementation.
| Item |
|---|
pub trait Driver |
HarnessError
Harness error type.
| Item |
|---|
pub enum HarnessError |
HarnessError :: fn driver(msg : impl Into <String>) -> Self |
HarnessError :: fn run_db(msg : impl Into <String>) -> Self |
HarnessError :: fn reporting(msg : impl Into <String>) -> Self |
HarnessError :: fn message(msg : impl Into <String>) -> Self |
HarnessError :: fn from(e : infrastructure_browser_automation::AutomationError) -> Self |
HarnessError :: fn from(e : infrastructure_browser_cdp::CdpError) -> Self |
HarnessError :: fn from(e : sqlx::Error) -> Self |
HarnessError :: fn from(e : operations_test_results::TestResultsError) -> Self |
Result
Harness error type.
| Item |
|---|
pub type Result<T>: std::result::Result <T, HarnessError> |
PublicRouteSmokeCheck
Worked example checks that exercise the engine end to end.
| Item |
|---|
pub struct PublicRouteSmokeCheck |
PublicRouteSmokeCheck :: fn new(name : impl Into <String>, path : impl Into <String>, expect_selector : impl Into <String>,) -> Self |
PublicRouteSmokeCheck :: fn name(& self) -> & str |
PublicRouteSmokeCheck :: fn tier(& self) -> Tier |
PublicRouteSmokeCheck :: fn surfaces(& self) -> Vec <String> |
PublicRouteSmokeCheck :: async fn verify(& self, ctx : & RunCtx) -> Result <CheckOutcome> |
StaffSurfaceCheckStub
Worked example checks that exercise the engine end to end.
| Item |
|---|
pub struct StaffSurfaceCheckStub |
StaffSurfaceCheckStub :: fn new(name : impl Into <String>) -> Self |
StaffSurfaceCheckStub :: fn name(& self) -> & str |
StaffSurfaceCheckStub :: fn tier(& self) -> Tier |
StaffSurfaceCheckStub :: fn surfaces(& self) -> Vec <String> |
StaffSurfaceCheckStub :: async fn populate(& self, _ctx : & RunCtx) -> Result <()> |
StaffSurfaceCheckStub :: async fn verify(& self, ctx : & RunCtx) -> Result <CheckOutcome> |
ForgeCmsSource
The real DefinitionSource — a projection of application-cms's public
| Item |
|---|
pub struct ForgeCmsSource |
ForgeCmsSource :: fn new() -> Self |
ForgeCmsSource :: fn templates(& self) -> Vec <DiscoveredDefinition> |
ForgeCmsSource :: fn blocks(& self) -> Vec <DiscoveredDefinition> |
ForgeCmsSource :: fn widgets(& self) -> Vec <DiscoveredDefinition> |
MutationCheck
The registry-driven generator — items 3 & 7 of the locked architecture.
| Item |
|---|
MutationCheck :: fn name(& self) -> & str |
MutationCheck :: fn tier(& self) -> Tier |
MutationCheck :: fn surfaces(& self) -> Vec <String> |
MutationCheck :: async fn verify(& self, ctx : & RunCtx) -> Result <CheckOutcome> |
PreviewChannel
The registry-driven generator — items 3 & 7 of the locked architecture.
| Item |
|---|
pub trait PreviewChannel |
fn generate_checks(source : Arc <dyn DefinitionSource>, channel : Arc <dyn PreviewChannel>,) -> Vec <Arc <dyn Check>> |
RenderCheck
The registry-driven generator — items 3 & 7 of the locked architecture.
| Item |
|---|
RenderCheck :: fn name(& self) -> & str |
RenderCheck :: fn tier(& self) -> Tier |
RenderCheck :: fn surfaces(& self) -> Vec <String> |
RenderCheck :: async fn verify(& self, ctx : & RunCtx) -> Result <CheckOutcome> |
kit_checks (other)
Reusable kit E2E checks — sprint 0.79 Phase 5, US-0.79.2 (T2).
| Item |
|---|
fn generated_kit_checks(channel : Arc <dyn crate::generator::PreviewChannel>,) -> Vec <Arc <dyn Check>> |
ComposerInteractionCheck
Reusable kit E2E checks — sprint 0.79 Phase 5, US-0.79.2 (T2).
| Item |
|---|
pub struct ComposerInteractionCheck |
ComposerInteractionCheck :: fn new(name : impl Into <String>, block_key : impl Into <String>, surface : Arc <dyn ComposerSurface>,) -> Self |
ComposerInteractionCheck :: fn name(& self) -> & str |
ComposerInteractionCheck :: fn tier(& self) -> Tier |
ComposerInteractionCheck :: fn surfaces(& self) -> Vec <String> |
ComposerInteractionCheck :: async fn verify(& self, ctx : & RunCtx) -> Result <CheckOutcome> |
ComposerSurface
Reusable kit E2E checks — sprint 0.79 Phase 5, US-0.79.2 (T2).
| Item |
|---|
pub trait ComposerSurface |
MediaAuthzAtUiCheck
Reusable kit E2E checks — sprint 0.79 Phase 5, US-0.79.2 (T2).
| Item |
|---|
pub struct MediaAuthzAtUiCheck |
MediaAuthzAtUiCheck :: fn new(name : impl Into <String>, private_asset_ids : impl IntoIterator <Item = String>, surface : Arc <dyn PublicPageSurface>,) -> Self |
MediaAuthzAtUiCheck :: fn name(& self) -> & str |
MediaAuthzAtUiCheck :: fn tier(& self) -> Tier |
MediaAuthzAtUiCheck :: fn surfaces(& self) -> Vec <String> |
MediaAuthzAtUiCheck :: async fn verify(& self, ctx : & RunCtx) -> Result <CheckOutcome> |
MediaPickerEligibilityCheck
Reusable kit E2E checks — sprint 0.79 Phase 5, US-0.79.2 (T2).
| Item |
|---|
pub struct MediaPickerEligibilityCheck |
MediaPickerEligibilityCheck :: fn new(name : impl Into <String>, public_field : TestId, gated_asset_id : impl Into <String>, public_asset_id : Option <String>, surface : Arc <dyn MediaPickerSurface>,) -> Self |
MediaPickerEligibilityCheck :: fn name(& self) -> & str |
MediaPickerEligibilityCheck :: fn tier(& self) -> Tier |
MediaPickerEligibilityCheck :: fn surfaces(& self) -> Vec <String> |
MediaPickerEligibilityCheck :: async fn verify(& self, ctx : & RunCtx) -> Result <CheckOutcome> |
MediaPickerSurface
Reusable kit E2E checks — sprint 0.79 Phase 5, US-0.79.2 (T2).
| Item |
|---|
pub trait MediaPickerSurface |
PublicPageSurface
Reusable kit E2E checks — sprint 0.79 Phase 5, US-0.79.2 (T2).
| Item |
|---|
pub trait PublicPageSurface |
LayerOutcome
The proactive assertion layers — Phase 4 (locked-architecture item 7).
| Item |
|---|
pub struct LayerOutcome |
LayerOutcome :: fn pass() -> Self |
LayerOutcome :: fn fail(detail : impl Into <String>) -> Self |
LayerOutcome :: fn note(detail : impl Into <String>) -> Self |
LayerOutcome :: fn with_findings(mut self, findings : impl IntoIterator <Item = Finding>) -> Self |
LayerOutcome :: fn with_metrics(mut self, metrics : impl IntoIterator <Item = MetricInput>) -> Self |
layers::a11y (other)
Layer 1 — a11y (axe): accessibility violations become findings.
| Item |
|---|
fn assert_a11y(audit : Option <& Value>) -> LayerOutcome |
layers::capture (other)
Layer 8 — console / network error capture: fold the driver's console and
| Item |
|---|
fn console_layer(entries : & Value, allow_substrings : & & str) -> LayerOutcome |
fn network_layer(entries : & Value, allow_substrings : & & str) -> LayerOutcome |
layers::flaky (other)
Layer 7 — flaky tracking: surface a check that passes then fails across
| Item |
|---|
fn classify_flake(statuses : & ResultStatus) -> FlakeStats |
fn assert_not_flaky(name : & str, history : & ResultStatus) -> LayerOutcome |
layers::i18n (other)
Layer 3 — i18n (ES/EN): render each surface in both locales and assert the
| Item |
|---|
fn leaked_i18n_keys(html : & str, key_prefixes : & & str) -> Vec <String> |
fn assert_i18n(es_html : & str, en_html : & str, expected_differ : & & str, key_prefixes : & & str,) -> LayerOutcome |
layers::perf (other)
Layer 6 — perf / Web-Vitals budgets: capture navigation/paint timings,
| Item |
|---|
fn assert_budget(name : & str, history : & f64, url : Option <& str>) -> LayerOutcome |
WebVitals
Layer 6 — perf / Web-Vitals budgets: capture navigation/paint timings,
| Item |
|---|
pub struct WebVitals |
WebVitals :: fn to_metrics(& self, url : Option <& str>) -> Vec <MetricInput> |
fn record_perf(vitals : & WebVitals, url : Option <& str>) -> LayerOutcome |
layers::pii (other)
Layer 9 — no-secret / PII-in-DOM guard — the SECURITY layer, and the one
| Item |
|---|
fn assert_no_servable_private_asset(public_html : & str, private_asset_ids : & String,) -> LayerOutcome |
fn scan_secrets_with(html : & str, policy : SecretScanPolicy) -> Vec <SecretHit> |
fn assert_no_secrets(public_html : & str) -> LayerOutcome |
fn assert_public_surface_clean(public_html : & str, private_asset_ids : & String,) -> LayerOutcome |
SecretHit
Layer 9 — no-secret / PII-in-DOM guard — the SECURITY layer, and the one
| Item |
|---|
pub struct SecretHit |
fn scan_secrets(html : & str) -> Vec <SecretHit> |
SecretScanPolicy
Layer 9 — no-secret / PII-in-DOM guard — the SECURITY layer, and the one
| Item |
|---|
pub struct SecretScanPolicy |
fn assert_no_secrets_with(public_html : & str, policy : SecretScanPolicy) -> LayerOutcome |
ServableLeak
Layer 9 — no-secret / PII-in-DOM guard — the SECURITY layer, and the one
| Item |
|---|
pub struct ServableLeak |
fn find_servable_leaks(html : & str, private_asset_ids : & String) -> Vec <ServableLeak> |
layers::state (other)
Layer 5 — error / empty / validation states: assert the right state
| Item |
|---|
fn classify_state(html : & str, signals : & StateSignals) -> PageState |
fn assert_state(html : & str, signals : & StateSignals, expected : PageState) -> LayerOutcome |
PageState
Layer 5 — error / empty / validation states: assert the right state
| Item |
|---|
pub enum PageState |
StateSignals
Layer 5 — error / empty / validation states: assert the right state
| Item |
|---|
pub struct StateSignals<'a> |
Viewport
Layer 4 — responsive viewports: assert layout invariants hold at desktop,
| Item |
|---|
pub enum Viewport |
Viewport :: fn dimensions(self) ->(u32, u32) |
Viewport :: fn label(self) -> & 'static str |
fn fold_viewports(results : & ViewportAssertion) -> LayerOutcome |
ViewportAssertion
Layer 4 — responsive viewports: assert layout invariants hold at desktop,
| Item |
|---|
pub struct ViewportAssertion |
ViewportAssertion :: fn pass(viewport : Viewport) -> Self |
ViewportAssertion :: fn fail(viewport : Viewport, detail : impl Into <String>) -> Self |
layers::visual (other)
Layer 2 — visual-regression: a screenshot per surface/viewport compared to
| Item |
|---|
fn compare_visual(baseline : & dyn VisualBaseline, key : & str, current : & Screenshot, tolerance : f64, masks : & MaskRegion,) -> VisualVerdict |
fn assert_visual(baseline : & dyn VisualBaseline, key : & str, current : & Screenshot, tolerance : f64, masks : & MaskRegion,) -> LayerOutcome |
MaskRegion
Layer 2 — visual-regression: a screenshot per surface/viewport compared to
| Item |
|---|
pub struct MaskRegion |
Screenshot
Layer 2 — visual-regression: a screenshot per surface/viewport compared to
| Item |
|---|
pub struct Screenshot |
Screenshot :: fn new(width : u32, height : u32, rgba : Vec <u8>) -> Result <Self, String> |
VisualBaseline
Layer 2 — visual-regression: a screenshot per surface/viewport compared to
| Item |
|---|
pub trait VisualBaseline |
VisualVerdict
Layer 2 — visual-regression: a screenshot per surface/viewport compared to
| Item |
|---|
pub enum VisualVerdict |
manifest_build (other)
Build the expected-control manifest from discovered definitions and
| Item |
|---|
pub const PAGE_WRITE_PERMISSION: & str |
fn expected_controls_for(def : & DiscoveredDefinition, source : & dyn DefinitionSource,) -> Vec <ExpectedControl> |
fn expected_controls_all(source : & dyn DefinitionSource) -> Vec <ExpectedControl> |
fn manifest_for <P : RolePolicy>(actor : Actor, def : & DiscoveredDefinition, source : & dyn DefinitionSource, policy : & P,) -> ExpectedControlManifest |
fn compare_manifest(manifest : & ExpectedControlManifest, observed : & ObservedControl,) -> Vec <ManifestMismatch> |
ManifestMismatch
Build the expected-control manifest from discovered definitions and
| Item |
|---|
pub enum ManifestMismatch |
ManifestMismatch :: fn test_id(& self) -> & str |
ObservedControl
Build the expected-control manifest from discovered definitions and
| Item |
|---|
pub struct ObservedControl |
ObservedControl :: fn present(test_id : impl Into <String>) -> Self |
ObservedControl :: fn disabled(test_id : impl Into <String>) -> Self |
CheckReport
Reporting: fold a run's per-check outcomes + findings into ONE run_id in
| Item |
|---|
pub struct CheckReport |
Finding
Reporting: fold a run's per-check outcomes + findings into ONE run_id in
| Item |
|---|
pub struct Finding |
Finding :: fn new(inner : CapturedFinding) -> Self |
Finding :: fn from_console(entries : & Value) ->(i64, Vec <Finding>) |
Finding :: fn from_network(entries : & Value) ->(i64, i64, Vec <Finding>) |
Finding :: fn from_axe(audit : & Value) ->(i64, Vec <Finding>) |
Finding :: fn note(kind : impl Into <String>, severity : impl Into <String>, detail : Value) -> Self |
Finding :: fn kind(& self) -> & str |
Finding :: fn severity(& self) -> & str |
FindingInput
Reporting: fold a run's per-check outcomes + findings into ONE run_id in
| Item |
|---|
FindingInput :: fn from(f : Finding) -> Self |
RunReport
Reporting: fold a run's per-check outcomes + findings into ONE run_id in
| Item |
|---|
pub struct RunReport |
RunReport :: fn run_status(& self) -> RunStatus |
RunReport :: fn into_complete_run_input(self) -> CompleteRunInput |
RunCtx
RunCtx — the per-run context handed to every check.
| Item |
|---|
pub struct RunCtx |
RunCtx :: fn new(run_id : Uuid, base_url : impl Into <String>, driver : Arc <dyn Driver>, db : Option <sqlx::PgPool>,) -> Self |
RunCtx :: fn run_id(& self) -> Uuid |
RunCtx :: fn base_url(& self) -> & str |
RunCtx :: fn url_for(& self, path : & str) -> String |
RunCtx :: fn driver(& self) -> & dyn Driver |
RunCtx :: fn db(& self) -> Option <& sqlx::PgPool> |
run_db (other)
Per-run database isolation via CREATE DATABASE … TEMPLATE.
| Item |
|---|
fn validate_identifier(kind : & 'static str, name : & str) -> Result <()> |
fn quote_identifier(name : & str) -> String |
fn run_db_name(run_id : Uuid) -> String |
fn swap_database_in_url(admin_url : & str, new_db : & str) -> Result <String> |
async fn database_exists(admin_url : & str, db_name : & str) -> Result <bool> |
RunDb
Per-run database isolation via CREATE DATABASE … TEMPLATE.
| Item |
|---|
pub struct RunDb |
RunDb :: async fn create_from_template(admin_url : & str, template_db : & str, run_id : Uuid,) -> Result <Self> |
RunDb :: fn db_name(& self) -> & str |
RunDb :: fn database_url(& self) -> Result <String> |
RunDb :: async fn connect_pool(& self) -> Result <PgPool> |
RunDb :: async fn drop_db(mut self) -> Result <()> |
RunDb :: fn drop(& mut self) |
RunConfig
The runner engine: register checks, filter by tier, execute (optionally in
| Item |
|---|
pub struct RunConfig |
RunConfig :: fn smoke(project : impl Into <String>) -> Self |
Runner
The runner engine: register checks, filter by tier, execute (optionally in
| Item |
|---|
pub struct Runner |
Runner :: fn new() -> Self |
Runner :: fn register(& mut self, check : Arc <dyn Check>) -> Result <()> |
Runner :: fn len(& self) -> usize |
Runner :: fn is_empty(& self) -> bool |
Runner :: fn scheduled(& self, tier : Tier) -> Vec <Arc <dyn Check>> |
Runner :: async fn run_to_report(& self, ctx : Arc <RunCtx>, config : & RunConfig) -> RunReport |
Runner :: async fn run(& self, ctx : Arc <RunCtx>, config : & RunConfig, service : & TestResultsService,) -> Result <(RunReport, Option <TestRun>)> |
How to use it
From `examples/consumer_suite.rs`:
use std::sync::Arc;
use async_trait::async_trait;
use operations_e2e_harness::{
Check, CheckOutcome, Finding, MediaAuthzAtUiCheck, PublicPageSurface, RunConfig, RunCtx,
Runner, Tier,
};
#[derive(Debug, Clone)]
pub struct Config {
pub base_url: String,
pub admin_email: String,
pub admin_password: String,
pub project: String,
}Module structure
operations_e2e_harness
adaptercheckcompletenesscontractcrawldiscoverydrivererrorexamplesforge_cms_sourcegeneratorkit_checkslayerslayers::a11ylayers::capturelayers::flakylayers::i18nlayers::perflayers::piilayers::statelayers::viewportlayers::visualmanifest_buildreportrun_ctxrun_dbrunner
_27 modules: past 25 the diagram stops being readable, so the tree above is the complete picture._
Public surface
`adapter`
| Item | What it is |
|---|---|
pub trait SaveReload | The surface-specific save → reload → read-rendered step an adapter needs to complete the round-trip proof |
fn selector_for(id : & TestId) -> String | Build the CSS selector that locates a control by its canonical test id |
fn known_value_for(id : & TestId) -> String | A known, distinctive value to drive into a control of a given kind, derived from the control's id so it is unique per control (so a "rendered shows" check can't be satisfied by some other control's value) |
pub trait FieldAdapter | One FieldKind test adapter: drives + asserts a single control kind, producing a MutationProof. |
pub struct TextInputAdapter | Adapter for ControlKind::TextInput (text / url fields). |
TextInputAdapter :: fn control(& self) -> ControlKind | — |
TextInputAdapter :: async fn drive(& self, driver : & dyn Driver, save_reload : & dyn SaveReload, id : & TestId, value : & str,) -> Result <MutationProof> | — |
pub struct TextareaAdapter | Adapter for ControlKind::Textarea (textarea / markdown fields). |
TextareaAdapter :: fn control(& self) -> ControlKind | — |
TextareaAdapter :: async fn drive(& self, driver : & dyn Driver, save_reload : & dyn SaveReload, id : & TestId, value : & str,) -> Result <MutationProof> | — |
pub struct SelectAdapter | Adapter for ControlKind::Select |
SelectAdapter :: fn control(& self) -> ControlKind | — |
SelectAdapter :: async fn drive(& self, driver : & dyn Driver, save_reload : & dyn SaveReload, id : & TestId, value : & str,) -> Result <MutationProof> | — |
pub struct MediaPickerAdapter | Adapter for ControlKind::MediaPicker (image fields) |
MediaPickerAdapter :: fn control(& self) -> ControlKind | — |
MediaPickerAdapter :: async fn drive(& self, driver : & dyn Driver, save_reload : & dyn SaveReload, id : & TestId, value : & str,) -> Result <MutationProof> | — |
fn adapter_for(control : ControlKind) -> Option <Box <dyn FieldAdapter>> | Dispatch to the adapter for a ControlKind |
async fn drive_field(driver : & dyn Driver, save_reload : & dyn SaveReload, id : & TestId, value : & str,) -> Result <Option <MutationProof>> | Drive a single field's mutation by dispatching to its adapter — the one entry point a generated check calls |
`check`
| Item | What it is |
|---|---|
pub enum Tier | When a check runs |
Tier :: fn runs_in(self, requested : Tier) -> bool | Whether a check of self tier should run when the runner is asked for requested |
pub struct CheckOutcome | What a single check produced: a pass/fail plus any findings |
CheckOutcome :: fn passed() -> Self | A passing outcome with no findings. |
CheckOutcome :: fn failed(detail : impl Into <String>) -> Self | A failing outcome carrying a detail message. |
CheckOutcome :: fn with_finding(mut self, finding : Finding) -> Self | Attach a finding (chainable). |
CheckOutcome :: fn with_findings(mut self, findings : impl IntoIterator <Item = Finding>) -> Self | Attach many findings (chainable). |
CheckOutcome :: fn with_metric(mut self, metric : MetricInput) -> Self | Attach a metric sample (chainable). |
CheckOutcome :: fn with_metrics(mut self, metrics : impl IntoIterator <Item = MetricInput>) -> Self | Attach many metric samples (chainable). |
CheckOutcome :: fn is_pass(& self) -> bool | Whether the check passed. |
CheckOutcome :: fn error(& self) -> Option <& str> | The failure detail, if any. |
CheckOutcome :: fn findings(& self) -> & Finding | The findings gathered by this check. |
CheckOutcome :: fn metrics(& self) -> & MetricInput | The metric samples gathered by this check. |
pub trait Check | One end-to-end check: idempotent populate, read-only verify, plus metadata |
`completeness`
| Item | What it is |
|---|---|
pub enum CompletenessGap | Why a definition is not yet coverable by the generator. |
CompletenessGap :: fn reason(& self) -> String | A human-readable explanation for the failing assertion. |
fn completeness_report(source : & dyn DefinitionSource) -> Vec <CompletenessGap> | Compute every completeness gap across a source |
fn assert_complete(source : & dyn DefinitionSource) | Assert a source is fully coverable, panicking with the collected gaps otherwise |
`contract`
| Item | What it is |
|---|---|
pub struct ProofLedger | Which of the three mandatory proofs a mutation has recorded |
pub enum Coverage | The verdict for a single mutation: covered (all three proofs) or incomplete (at least one missing), with the first missing proof named for the failure message. |
pub enum MissingProof | The specific proof a mutation failed to establish — the actionable detail in a no-op-theater failure. |
MissingProof :: fn reason(self) -> & 'static str | A human-readable reason for the failure message. |
pub struct MutationProof | A witness that a mutation proved (or failed to prove) the three-part contract |
MutationProof :: fn new(target : impl Into <String>, set_value : impl Into <String>) -> Self | Start a proof for target with the value set_value that will be driven into it |
MutationProof :: fn target(& self) -> & str | The control id under proof. |
MutationProof :: fn set_value(& self) -> & str | The value that was driven into the control. |
MutationProof :: fn dom_reflected(mut self, read_back : & str) -> Self | Record proof (a): the DOM control reflected the set value |
MutationProof :: fn survived_reload(mut self, read_after_reload : & str) -> Self | Record proof (b): the value survived save + reload |
MutationProof :: fn rendered_surface(mut self, shown : bool) -> Self | Record proof (c): the rendered surface showed the value |
MutationProof :: fn ledger(& self) -> ProofLedger | The ledger so far (for tests / diagnostics). |
MutationProof :: fn into_coverage(self) -> Coverage | Collapse to a verdict |
MutationProof :: fn is_covered(& self) -> bool | Convenience: true iff MutationProof::into_coverage is Coverage::Covered. |
`crawl`
| Item | What it is |
|---|---|
pub struct PageRecord | — |
pub struct CrawlPolicy | Configuration for what counts as an authored value worth asserting. |
CrawlPolicy :: fn default() -> Self | — |
pub struct ExpectedContent | The set of authored values a page is expected to render, extracted from its metadata by expected_rendered_values. |
fn expected_rendered_values(metadata : & Value, policy : & CrawlPolicy) -> ExpectedContent | Walk a page's metadata and collect the authored leaf values that should be visible in the rendered page |
pub struct MissingContent | An authored value that did NOT appear in the rendered page — a content-render failure. |
fn missing_content(expected : & ExpectedContent, rendered_html : & str) -> Vec <MissingContent> | Compare the expected authored content against the rendered HTML, returning the authored values that are missing |
fn rendered_contains(rendered_html : & str, value : & str) -> bool | Whether rendered_html contains the authored value, accounting for asset refs (matched by their resolvable identifier rather than the raw scheme) |
`discovery`
| Item | What it is |
|---|---|
pub struct DiscoveredField | One field of a discovered definition — the projection of application-cms's FieldDefinition the generator needs |
DiscoveredField :: fn leaf(name : impl Into <String>, kind : FieldKind) -> Self | A leaf field (no children, no options). |
DiscoveredField :: fn select(name : impl Into <String>, options : impl IntoIterator <Item = String>) -> Self | A select field with options. |
DiscoveredField :: fn repeating(name : impl Into <String>, item_fields : impl IntoIterator <Item = DiscoveredField>,) -> Self | A repeating field with a per-row sub-form. |
DiscoveredField :: fn is_container(& self) -> bool | Whether this field is a container (its control holds child controls the generator must recurse into) — drives leaf-vs-container handling. |
pub struct DiscoveredDefinition | One discovered definition — a template, block, or widget registry entry projected to exactly what the generator needs. |
DiscoveredDefinition :: fn has_sample(& self) -> bool | Whether this definition ships a non-empty sample fixture |
pub trait DefinitionSource | The seam the generator discovers from |
`driver`
| Item | What it is |
|---|---|
pub trait Driver | The browser surface a check is allowed to touch |
pub struct CdpDriver | Live driver: a thin wrapper over infrastructure-browser-automation |
CdpDriver :: fn connect(cdp_base_url : & str) -> Result <Self> | Connect to a Chrome DevTools endpoint (e.g |
CdpDriver :: async fn navigate(& self, url : & str) -> Result <()> | — |
CdpDriver :: async fn read_text(& self, css_selector : & str) -> Result <String> | — |
CdpDriver :: async fn read_value(& self, css_selector : & str) -> Result <String> | — |
CdpDriver :: async fn page_html(& self) -> Result <String> | — |
CdpDriver :: async fn click(& self, css_selector : & str) -> Result <()> | — |
CdpDriver :: async fn type_text(& self, css_selector : & str, text : & str) -> Result <()> | — |
CdpDriver :: async fn exists(& self, css_selector : & str) -> Result <bool> | — |
CdpDriver :: async fn take_console(& self) -> Result <Vec <Value>> | — |
CdpDriver :: async fn take_network(& self) -> Result <Vec <Value>> | — |
CdpDriver :: async fn run_axe(& self) -> Result <Option <Value>> | — |
CdpDriver :: async fn reset_session(& self) -> Result <()> | — |
`error`
| Item | What it is |
|---|---|
pub type Result<T>: std::result::Result <T, HarnessError> | The result alias used throughout the harness. |
pub enum HarnessError | Everything that can go wrong while populating, verifying, or reporting a run. |
HarnessError :: fn driver(msg : impl Into <String>) -> Self | Wrap a driver-layer message. |
HarnessError :: fn run_db(msg : impl Into <String>) -> Self | Wrap a run-db-layer message. |
HarnessError :: fn reporting(msg : impl Into <String>) -> Self | Wrap a reporting-layer message. |
HarnessError :: fn message(msg : impl Into <String>) -> Self | A free-form check-authoring message (use inside populate/verify). |
HarnessError :: fn from(e : infrastructure_browser_automation::AutomationError) -> Self | — |
HarnessError :: fn from(e : infrastructure_browser_cdp::CdpError) -> Self | — |
HarnessError :: fn from(e : sqlx::Error) -> Self | — |
HarnessError :: fn from(e : operations_test_results::TestResultsError) -> Self | — |
`examples`
| Item | What it is |
|---|---|
pub struct PublicRouteSmokeCheck | A smoke check: anonymously GET a public route and assert it loaded with no console errors |
PublicRouteSmokeCheck :: fn new(name : impl Into <String>, path : impl Into <String>, expect_selector : impl Into <String>,) -> Self | Build a smoke check for path, expecting expect_selector to render. |
PublicRouteSmokeCheck :: fn name(& self) -> & str | — |
PublicRouteSmokeCheck :: fn tier(& self) -> Tier | — |
PublicRouteSmokeCheck :: fn surfaces(& self) -> Vec <String> | — |
PublicRouteSmokeCheck :: async fn verify(& self, ctx : & RunCtx) -> Result <CheckOutcome> | — |
pub struct StaffSurfaceCheckStub | A staff-surface check stub: demonstrates the populate (idempotent setup) plus DB-backed verify shape without standing up real auth (that is a bespoke hand-written flow for a later phase) |
StaffSurfaceCheckStub :: fn new(name : impl Into <String>) -> Self | Build the stub. |
StaffSurfaceCheckStub :: fn name(& self) -> & str | — |
StaffSurfaceCheckStub :: fn tier(& self) -> Tier | — |
StaffSurfaceCheckStub :: fn surfaces(& self) -> Vec <String> | — |
StaffSurfaceCheckStub :: async fn populate(& self, _ctx : & RunCtx) -> Result <()> | — |
StaffSurfaceCheckStub :: async fn verify(& self, ctx : & RunCtx) -> Result <CheckOutcome> | — |
`forge_cms_source`
| Item | What it is |
|---|---|
pub struct ForgeCmsSource | The application-cms-backed definition source |
ForgeCmsSource :: fn new() -> Self | Construct the source. |
ForgeCmsSource :: fn templates(& self) -> Vec <DiscoveredDefinition> | — |
ForgeCmsSource :: fn blocks(& self) -> Vec <DiscoveredDefinition> | — |
ForgeCmsSource :: fn widgets(& self) -> Vec <DiscoveredDefinition> | — |
`generator`
| Item | What it is |
|---|---|
pub trait PreviewChannel | The surface-specific render + save/reload channel a generated check drives |
fn generate_checks(source : Arc <dyn DefinitionSource>, channel : Arc <dyn PreviewChannel>,) -> Vec <Arc <dyn Check>> | Generate the full set of checks for a source: a render check per definition plus a mutation check per drivable leaf control |
RenderCheck :: fn name(& self) -> & str | — |
RenderCheck :: fn tier(& self) -> Tier | — |
RenderCheck :: fn surfaces(& self) -> Vec <String> | — |
RenderCheck :: async fn verify(& self, ctx : & RunCtx) -> Result <CheckOutcome> | — |
MutationCheck :: fn name(& self) -> & str | — |
MutationCheck :: fn tier(& self) -> Tier | — |
MutationCheck :: fn surfaces(& self) -> Vec <String> | — |
MutationCheck :: async fn verify(& self, ctx : & RunCtx) -> Result <CheckOutcome> | — |
`kit_checks`
| Item | What it is |
|---|---|
fn generated_kit_checks(channel : Arc <dyn crate::generator::PreviewChannel>,) -> Vec <Arc <dyn Check>> | Generate the reusable kit checks from the real application-cms registries |
pub trait ComposerSurface | The consumer-supplied seam for driving the block/widget composer — the bits the harness cannot know generically (which palette entry adds a block, where the Save button is, how to read the rendered order back) |
pub struct ComposerInteractionCheck | Hand-written check: the block composer's palette-add + reorder flow |
ComposerInteractionCheck :: fn new(name : impl Into <String>, block_key : impl Into <String>, surface : Arc <dyn ComposerSurface>,) -> Self | Build the composer interaction check. |
ComposerInteractionCheck :: fn name(& self) -> & str | — |
ComposerInteractionCheck :: fn tier(& self) -> Tier | — |
ComposerInteractionCheck :: fn surfaces(& self) -> Vec <String> | — |
ComposerInteractionCheck :: async fn verify(& self, ctx : & RunCtx) -> Result <CheckOutcome> | — |
pub trait MediaPickerSurface | The consumer-supplied seam for the media picker opened against a given page field |
pub struct MediaPickerEligibilityCheck | Hand-written check: a gated asset is NOT selectable for a public-page field |
MediaPickerEligibilityCheck :: fn new(name : impl Into <String>, public_field : TestId, gated_asset_id : impl Into <String>, public_asset_id : Option <String>, surface : Arc <dyn MediaPickerSurface>,) -> Self | Build the eligibility check |
MediaPickerEligibilityCheck :: fn name(& self) -> & str | — |
MediaPickerEligibilityCheck :: fn tier(& self) -> Tier | — |
MediaPickerEligibilityCheck :: fn surfaces(& self) -> Vec <String> | — |
MediaPickerEligibilityCheck :: async fn verify(& self, ctx : & RunCtx) -> Result <CheckOutcome> | — |
pub trait PublicPageSurface | The consumer-supplied seam for fetching a composed public page's rendered HTML as an anonymous visitor. |
pub struct MediaAuthzAtUiCheck | Hand-written check: the media-authz invariant on a real composed PUBLIC page (0.77 / 0.80) |
MediaAuthzAtUiCheck :: fn new(name : impl Into <String>, private_asset_ids : impl IntoIterator <Item = String>, surface : Arc <dyn PublicPageSurface>,) -> Self | Build the media-authz check |
MediaAuthzAtUiCheck :: fn name(& self) -> & str | — |
MediaAuthzAtUiCheck :: fn tier(& self) -> Tier | — |
MediaAuthzAtUiCheck :: fn surfaces(& self) -> Vec <String> | — |
MediaAuthzAtUiCheck :: async fn verify(& self, ctx : & RunCtx) -> Result <CheckOutcome> | — |
`layers`
| Item | What it is |
|---|---|
pub struct LayerOutcome | The uniform result of running one layer: a verdict plus what it observed |
LayerOutcome :: fn pass() -> Self | A passing (assertion-held) layer outcome. |
LayerOutcome :: fn fail(detail : impl Into <String>) -> Self | A failing layer outcome with a detail message. |
LayerOutcome :: fn note(detail : impl Into <String>) -> Self | A passing outcome carrying an informational detail (e.g |
LayerOutcome :: fn with_findings(mut self, findings : impl IntoIterator <Item = Finding>) -> Self | Attach findings (chainable). |
LayerOutcome :: fn with_metrics(mut self, metrics : impl IntoIterator <Item = MetricInput>) -> Self | Attach metric samples (chainable). |
`layers::a11y`
| Item | What it is |
|---|---|
fn assert_a11y(audit : Option <& Value>) -> LayerOutcome | Assert accessibility on a surface from its axe audit object |
`layers::capture`
| Item | What it is |
|---|---|
fn console_layer(entries : & Value, allow_substrings : & & str) -> LayerOutcome | Fold the console buffer into findings and fail on any non-allowlisted error |
fn network_layer(entries : & Value, allow_substrings : & & str) -> LayerOutcome | Fold the network buffer into findings and fail on any non-allowlisted failure (4xx/5xx or transport error) |
`layers::flaky`
| Item | What it is |
|---|---|
fn classify_flake(statuses : & ResultStatus) -> FlakeStats | Classify a check from its newest-first result history (the order recent_results returns) |
fn assert_not_flaky(name : & str, history : & ResultStatus) -> LayerOutcome | Run the flaky layer for a check given its prior result history |
`layers::i18n`
| Item | What it is |
|---|---|
fn leaked_i18n_keys(html : & str, key_prefixes : & & str) -> Vec <String> | Detect raw i18n keys / unrendered placeholders leaking into rendered HTML |
fn assert_i18n(es_html : & str, en_html : & str, expected_differ : & & str, key_prefixes : & & str,) -> LayerOutcome | Assert i18n correctness given the ES and EN rendered HTML |
`layers::perf`
| Item | What it is |
|---|---|
pub struct WebVitals | The standard Web-Vitals + navigation timings, in milliseconds (CLS is a unitless score) |
WebVitals :: fn to_metrics(& self, url : Option <& str>) -> Vec <MetricInput> | Project the captured vitals into MetricInput samples (skipping the ones not captured) |
fn record_perf(vitals : & WebVitals, url : Option <& str>) -> LayerOutcome | Emit the captured vitals as metric samples on the outcome (always), with no budget verdict |
fn assert_budget(name : & str, history : & f64, url : Option <& str>) -> LayerOutcome | Check ONE metric's newest value against an adaptive budget derived from its history, AND emit the newest value as a metric sample |
`layers::pii`
| Item | What it is |
|---|---|
pub struct ServableLeak | A detected leak of a private asset's servable URL on a public surface. |
fn assert_no_servable_private_asset(public_html : & str, private_asset_ids : & String,) -> LayerOutcome | Assert that NO private asset emits a servable URL on this public surface |
fn find_servable_leaks(html : & str, private_asset_ids : & String) -> Vec <ServableLeak> | Find every private asset id that appears inside a servable media URL. |
pub struct SecretHit | A detected secret / token / PII hit in public DOM. |
pub struct SecretScanPolicy | Policy for scan_secrets: which classes to scan for |
fn scan_secrets(html : & str) -> Vec <SecretHit> | Scan public HTML for embedded secrets / tokens (and, when the policy opts in, email PII), with SecretScanPolicy::default |
fn scan_secrets_with(html : & str, policy : SecretScanPolicy) -> Vec <SecretHit> | Scan with an explicit SecretScanPolicy (e.g |
fn assert_no_secrets(public_html : & str) -> LayerOutcome | Assert no secret / token in public DOM, failing on any hit |
fn assert_no_secrets_with(public_html : & str, policy : SecretScanPolicy) -> LayerOutcome | Assert no secret in public DOM under an explicit SecretScanPolicy. |
fn assert_public_surface_clean(public_html : & str, private_asset_ids : & String,) -> LayerOutcome | Run BOTH security assertions on a public surface in one call. |
`layers::state`
| Item | What it is |
|---|---|
pub enum PageState | Which UI state a surface is currently showing, as classified from the render. |
pub struct StateSignals<'a> | The signals available to classify a state, beyond the raw HTML. |
fn classify_state(html : & str, signals : & StateSignals) -> PageState | Classify the rendered state of a page from its HTML + signals |
fn assert_state(html : & str, signals : & StateSignals, expected : PageState) -> LayerOutcome | Assert the surface rendered the expected state |
`layers::viewport`
| Item | What it is |
|---|---|
pub enum Viewport | A standard responsive breakpoint with its emulated device-metric dimensions. |
Viewport :: fn dimensions(self) ->(u32, u32) | (width, height) in CSS pixels for Emulation.setDeviceMetricsOverride. |
Viewport :: fn label(self) -> & 'static str | A stable label for keys / reporting ("desktop" / "tablet" / "mobile"). |
pub struct ViewportAssertion | The result of one viewport's layout assertion: did the surface pass at this size, and why not if it failed. |
ViewportAssertion :: fn pass(viewport : Viewport) -> Self | A passing result for a viewport. |
ViewportAssertion :: fn fail(viewport : Viewport, detail : impl Into <String>) -> Self | A failing result for a viewport with detail. |
fn fold_viewports(results : & ViewportAssertion) -> LayerOutcome | Fold a set of per-viewport assertion results into one LayerOutcome, each failure attributed to its viewport (label in the detail + a per-viewport finding) |
`layers::visual`
| Item | What it is |
|---|---|
pub struct MaskRegion | A rectangular region (in pixels) whose contents are volatile and must be neutralised before diffing, so it can never cause a false drift. |
pub struct Screenshot | A captured screenshot: raw RGBA pixels plus its dimensions |
Screenshot :: fn new(width : u32, height : u32, rgba : Vec <u8>) -> Result <Self, String> | Build a screenshot, validating the buffer length matches the dimensions. |
pub enum VisualVerdict | The verdict of a visual comparison. |
pub trait VisualBaseline | Persisted baselines, keyed by a stable surface+viewport key |
fn compare_visual(baseline : & dyn VisualBaseline, key : & str, current : & Screenshot, tolerance : f64, masks : & MaskRegion,) -> VisualVerdict | Compare current against the baseline for key, masking masks in both |
fn assert_visual(baseline : & dyn VisualBaseline, key : & str, current : & Screenshot, tolerance : f64, masks : & MaskRegion,) -> LayerOutcome | Run the visual layer for a surface: compare and translate the verdict to a LayerOutcome |
`manifest_build`
| Item | What it is |
|---|---|
pub const PAGE_WRITE_PERMISSION: & str | The permission gate every CMS field control sits behind — the truthful value the render engine stamps as data-test-perm (application-rbac's cms.pages.write route gate) |
fn expected_controls_for(def : & DiscoveredDefinition, source : & dyn DefinitionSource,) -> Vec <ExpectedControl> | Build the flat list of ExpectedControls for ONE definition, predicting the exact TestId the render engine emits for each leaf control |
fn expected_controls_all(source : & dyn DefinitionSource) -> Vec <ExpectedControl> | Build expected controls across EVERY definition in a source — the whole expected manifest universe before per-actor resolution. |
fn manifest_for <P : RolePolicy>(actor : Actor, def : & DiscoveredDefinition, source : & dyn DefinitionSource, policy : & P,) -> ExpectedControlManifest | Resolve a per-actor ExpectedControlManifest for one definition. |
pub enum ManifestMismatch | One way the live DOM disagreed with the expected-control manifest. |
ManifestMismatch :: fn test_id(& self) -> & str | The offending test-id string. |
pub struct ObservedControl | What the harness observed for one control in the DOM — the input to compare_manifest |
ObservedControl :: fn present(test_id : impl Into <String>) -> Self | An observed, interactable control. |
ObservedControl :: fn disabled(test_id : impl Into <String>) -> Self | An observed, disabled control. |
fn compare_manifest(manifest : & ExpectedControlManifest, observed : & ObservedControl,) -> Vec <ManifestMismatch> | Compare a per-actor ExpectedControlManifest against the controls observed in the DOM, returning every mismatch (empty ⇒ the DOM matches the oracle for this actor) |
`report`
| Item | What it is |
|---|---|
pub struct Finding | A single finding (console error, network failure, axe violation, or a check-authored note), in the framework's canonical shape. |
Finding :: fn new(inner : CapturedFinding) -> Self | Wrap a CapturedFinding produced by the capture mappers. |
Finding :: fn from_console(entries : & Value) ->(i64, Vec <Finding>) | Build the console findings for a run from the driver's console buffer, reusing capture::console_findings |
Finding :: fn from_network(entries : & Value) ->(i64, i64, Vec <Finding>) | Build the network-failure findings from the driver's network buffer, reusing capture::network_failures |
Finding :: fn from_axe(audit : & Value) ->(i64, Vec <Finding>) | Build the axe findings from an axe audit object, reusing capture::axe_findings |
Finding :: fn note(kind : impl Into <String>, severity : impl Into <String>, detail : Value) -> Self | A check-authored finding (e.g |
Finding :: fn kind(& self) -> & str | The finding kind ("console", "network", "accessibility", …). |
Finding :: fn severity(& self) -> & str | The normalized severity ("info" \ |
FindingInput :: fn from(f : Finding) -> Self | — |
pub struct CheckReport | The per-check row the runner assembles before reporting |
pub struct RunReport | Everything needed to record one run. |
RunReport :: fn run_status(& self) -> RunStatus | The overall run status: Passed iff every check passed, else Failed. |
RunReport :: fn into_complete_run_input(self) -> CompleteRunInput | Build the CompleteRunInput for record_complete_run |
`run_ctx`
| Item | What it is |
|---|---|
pub struct RunCtx | Per-run context shared by all checks in a run. |
RunCtx :: fn new(run_id : Uuid, base_url : impl Into <String>, driver : Arc <dyn Driver>, db : Option <sqlx::PgPool>,) -> Self | Build a context for a run |
RunCtx :: fn run_id(& self) -> Uuid | The run id. |
RunCtx :: fn base_url(& self) -> & str | The base URL of the server under test (no trailing slash guaranteed by the caller; url_for normalizes joins). |
RunCtx :: fn url_for(& self, path : & str) -> String | Join a path onto the base URL, collapsing a doubled slash at the seam. |
RunCtx :: fn driver(& self) -> & dyn Driver | The browser driver for this run. |
RunCtx :: fn db(& self) -> Option <& sqlx::PgPool> | The per-run DB pool, if one was provisioned |
`run_db`
| Item | What it is |
|---|---|
fn validate_identifier(kind : & 'static str, name : & str) -> Result <()> | Validate a Postgres identifier we intend to interpolate into DDL |
fn quote_identifier(name : & str) -> String | Double-quote an identifier for safe interpolation, doubling any embedded quote per the SQL standard |
fn run_db_name(run_id : Uuid) -> String | Derive a unique, valid run-database name from a run id |
fn swap_database_in_url(admin_url : & str, new_db : & str) -> Result <String> | Replace the database in a Postgres connection URL, preserving everything else (user, password, host, port, query string) |
pub struct RunDb | A live per-run database cloned from a template |
RunDb :: async fn create_from_template(admin_url : & str, template_db : & str, run_id : Uuid,) -> Result <Self> | Clone template_db into a fresh per-run database |
RunDb :: fn db_name(& self) -> & str | The clone's database name. |
RunDb :: fn database_url(& self) -> Result <String> | A connection URL pointing at the clone, derived from the admin URL. |
RunDb :: async fn connect_pool(& self) -> Result <PgPool> | Open a pool against the clone (for the per-run server and DB-level assertions). |
RunDb :: async fn drop_db(mut self) -> Result <()> | Checked teardown: terminate stragglers on the clone, then drop it |
RunDb :: fn drop(& mut self) | — |
async fn database_exists(admin_url : & str, db_name : & str) -> Result <bool> | Confirm a database exists (used by live tests to assert clone/drop). |
`runner`
| Item | What it is |
|---|---|
pub struct RunConfig | Knobs for a single run. |
RunConfig :: fn smoke(project : impl Into <String>) -> Self | A minimal smoke config: serial, reporting on, nil tenant. |
pub struct Runner | A registry of checks plus the wiring to run them |
Runner :: fn new() -> Self | An empty runner. |
Runner :: fn register(& mut self, check : Arc <dyn Check>) -> Result <()> | Register a check |
Runner :: fn len(& self) -> usize | Number of registered checks. |
Runner :: fn is_empty(& self) -> bool | Whether no checks are registered. |
Runner :: fn scheduled(& self, tier : Tier) -> Vec <Arc <dyn Check>> | The checks that would run for a given tier, in registration order. |
Runner :: async fn run_to_report(& self, ctx : Arc <RunCtx>, config : & RunConfig) -> RunReport | Run the scheduled checks against ctx, building the run report |
Runner :: async fn run(& self, ctx : Arc <RunCtx>, config : & RunConfig, service : & TestResultsService,) -> Result <(RunReport, Option <TestRun>)> | Run, then record to the test-results store unless no_report |
Re-exports. Exported here, defined elsewhere.
| Export | Defined in |
|---|---|
RunCtx | run_ctx::RunCtx |
generated_kit_checks | kit_checks::generated_kit_checks |
{CdpDriver,Driver} | driver::{CdpDriver,Driver} |
{Check,CheckOutcome,Tier} | check::{Check,CheckOutcome,Tier} |
{CheckReport,Finding,RunReport} | report::{CheckReport,Finding,RunReport} |
{ComposerInteractionCheck,ComposerSurface,MediaAuthzAtUiCheck,MediaPickerEligibilityCheck,MediaPickerSurface,PublicPageSurface,} | kit_checks::{ComposerInteractionCheck,ComposerSurface,MediaAuthzAtUiCheck,MediaPickerEligibilityCheck,MediaPickerSurface,PublicPageSurface,} |
{Coverage,MissingProof,MutationProof,ProofLedger} | contract::{Coverage,MissingProof,MutationProof,ProofLedger} |
{DefinitionSource,DiscoveredDefinition,DiscoveredField} | discovery::{DefinitionSource,DiscoveredDefinition,DiscoveredField} |
{HarnessError,Result} | error::{HarnessError,Result} |
{RunConfig,Runner} | runner::{RunConfig,Runner} |
{adapter_for,drive_field,known_value_for,selector_for,FieldAdapter,SaveReload,} | adapter::{adapter_for,drive_field,known_value_for,selector_for,FieldAdapter,SaveReload,} |
{assert_complete,completeness_report,CompletenessGap} | completeness::{assert_complete,completeness_report,CompletenessGap} |
{assert_visual,compare_visual,MaskRegion,Screenshot,VisualBaseline,VisualVerdict,},LayerOutcome,} | layers::{a11y::assert_a11y,capture::{console_layer,network_layer},flaky::{assert_not_flaky,classify_flake},i18n::{assert_i18n,leaked_i18n_keys},perf::{assert_budget,record_perf,WebVitals},pii::{assert_no_secrets,assert_no_secrets_with,assert_no_servable_private_asset,assert_public_surface_clean,find_servable_leaks,scan_secrets,scan_secrets_with,SecretHit,SecretScanPolicy,ServableLeak,},state::{assert_state,classify_state,PageState,StateSignals},viewport::{fold_viewports,Viewport,ViewportAssertion},visual::{assert_visual,compare_visual,MaskRegion,Screenshot,VisualBaseline,VisualVerdict,},LayerOutcome,} |
{compare_manifest,expected_controls_all,expected_controls_for,manifest_for,ManifestMismatch,ObservedControl,PAGE_WRITE_PERMISSION,} | manifest_build::{compare_manifest,expected_controls_all,expected_controls_for,manifest_for,ManifestMismatch,ObservedControl,PAGE_WRITE_PERMISSION,} |
{database_exists,quote_identifier,run_db_name,swap_database_in_url,validate_identifier,RunDb,} | run_db::{database_exists,quote_identifier,run_db_name,swap_database_in_url,validate_identifier,RunDb,} |
{expected_rendered_values,missing_content,rendered_contains,CrawlPolicy,ExpectedContent,MissingContent,PageRecord,} | crawl::{expected_rendered_values,missing_content,rendered_contains,CrawlPolicy,ExpectedContent,MissingContent,PageRecord,} |
{generate_checks,PreviewChannel} | generator::{generate_checks,PreviewChannel} |
Boundary
Reaches into application, infrastructure.
Shares tier operations with 40 other crates: operations-approval-workflow, operations-assessments, operations-block-imaging, operations-boot-media, operations-browser-agent-worker, operations-camera-discovery, operations-camera-liveview, operations-camera-registry, … (40 total).
_What this crate deliberately does NOT own is a judgment. No committed registry records one for it, so none is stated here._
Where it sits
| Tier (ontology) | operations |
| Architectural role (taxonomy) | unclassified (baselined) |
| Location | crates/operations/e2e-harness |
| Vocabulary in force (lexicon) | current |
Tier flow. Which tiers this crate's own edges cross.
flowchart LR n_operations["operations"] --> n_application["application"] n_operations["operations"] --> n_infrastructure["infrastructure"]
Dependencies
Runtime, in this workspace.
| Crate | Tier | Optional | Only on |
|---|---|---|---|
| `application-cms` | application | yes | always |
| `application-test-results` | application | no | always |
| `infrastructure-browser-automation` | infrastructure | no | always |
| `infrastructure-browser-cdp` | infrastructure | no | always |
| `operations-e2e-contract` | operations | no | always |
| `operations-test-results` | operations | no | always |
Runtime, from outside the workspace.
| Crate | Requirement | Features | Optional | Only on |
|---|---|---|---|---|
async-trait | ^0.1 | — | no | always |
chrono | ^0.4 | serde | no | always |
serde | ^1 | derive | no | always |
serde_json | ^1 | — | no | always |
sqlx | ^0.8 | runtime-tokio, postgres, chrono, uuid, json | no | always |
thiserror | ^2 | — | no | always |
tokio | ^1 | full | no | always |
tracing | ^0.1 | — | no | always |
uuid | ^1 | v4, v7, serde, js | no | always |
Development, from outside the workspace.
| Crate | Requirement | Features | Optional | Only on |
|---|---|---|---|---|
tokio-test | ^0.4 | — | no | always |
Build. None.
Depended on by. Nothing in this workspace.
Signal flow — what reaches this crate, and what it reaches.
flowchart LR SELF["operations-e2e-harness"] SELF -->|runtime| n_application_cms["application-cms"] SELF -->|runtime| n_application_test_results["application-test-results"] SELF -->|runtime| n_infrastructure_browser_automation["infrastructure-browser-automation"] SELF -->|runtime| n_infrastructure_browser_cdp["infrastructure-browser-cdp"] SELF -->|runtime| n_operations_e2e_contract["operations-e2e-contract"] SELF -->|runtime| n_operations_test_results["operations-test-results"] classDef self fill:#1f883d,stroke:#1f883d,color:#fff; class SELF self;
Feature flags
| Feature | Enables | On by default |
|---|---|---|
default | — | yes |
forge-cms-source | dep:application-cms | no |
flowchart LR n_default["default"] n_forge_cms_source["forge-cms-source"] --> n_dep_application_cms["dep:application-cms"]
Targets
| Kind | Name | Source |
|---|---|---|
| example | consumer_suite | `examples/consumer_suite.rs` |
| lib | operations_e2e_harness | `src/lib.rs` |
| test | fake_driver | `tests/fake_driver.rs` |
| test | generator_tests | `tests/generator_tests.rs` |
| test | kit_checks_tests | `tests/kit_checks_tests.rs` |
| test | layer_tests | `tests/layer_tests.rs` |
| test | live_browser_tests | `tests/live_browser_tests.rs` |
| test | run_db_tests | `tests/run_db_tests.rs` |
| test | runner_tests | `tests/runner_tests.rs` |
Error model
| Error type | Named by |
|---|---|
HarnessError | Result |
Operational characteristics
| Property | Evidence |
|---|---|
| async public surface | yes |
| async runtime | yes |
| database access | yes |
| network I/O | none detected |
| unsafe code | none detected |
| environment variables | none detected |
No unsafe block, unsafe fn, unsafe impl or unsafe trait was found by the parser anywhere in this crate's source.
Configuration
No environment variable is read with a literal name anywhere in this crate. A variable whose key is computed at run time cannot be listed here, and is not claimed to be absent.
Related capabilities
No workspace crate depends on this one.
Verification
| Kind | Count |
|---|---|
| Unit tests | 102 |
| Integration tests | 41 |
| Examples | 1 |
| Doctests | 0 |
Evidence by module. How often each public module is named by something executable.
| Module | Tests | Examples | Consumers |
|---|---|---|---|
adapter | 10 | 0 | 0 |
check | 3 | 3 | 0 |
completeness | 3 | 0 | 0 |
contract | 4 | 0 | 0 |
crawl | 7 | 0 | 0 |
discovery | 3 | 0 | 0 |
driver | 2 | 2 | 0 |
error | 2 | 1 | 0 |
examples | 2 | 0 | 0 |
forge_cms_source | 1 | 0 | 0 |
generator | 2 | 0 | 0 |
kit_checks | 7 | 5 | 0 |
layers | 1 | 0 | 0 |
layers::a11y | 1 | 0 | 0 |
layers::capture | 2 | 0 | 0 |
layers::flaky | 2 | 0 | 0 |
layers::i18n | 2 | 0 | 0 |
layers::perf | 3 | 0 | 0 |
layers::pii | 10 | 0 | 0 |
layers::state | 4 | 0 | 0 |
layers::viewport | 3 | 0 | 0 |
layers::visual | 6 | 0 | 0 |
manifest_build | 7 | 0 | 0 |
report | 3 | 1 | 0 |
run_ctx | 1 | 1 | 0 |
run_db | 6 | 1 | 0 |
runner | 2 | 2 | 0 |
What the tests establish, by name:
allowed_actor_clean_dom_has_no_mismatches_end_to_end—tests/generator_tests.rsdenial_path_anonymous_leak_is_caught_end_to_end—tests/generator_tests.rsgenerated_mutation_check_fails_on_no_op_theater—tests/generator_tests.rsgenerated_mutation_check_fails_when_input_is_swallowed—tests/generator_tests.rsgenerated_mutation_check_is_covered_when_surface_renders—tests/generator_tests.rsgenerated_render_check_fails_when_a_control_is_missing—tests/generator_tests.rsgenerated_render_check_passes_when_controls_present—tests/generator_tests.rsgenerator_emits_render_plus_mutation_checks_in_one_runner—tests/generator_tests.rsshared_fake_driver_supports_read_value—tests/generator_tests.rsmedia_authz_kit_check_fails_when_a_private_asset_leaks—tests/kit_checks_tests.rsmedia_authz_kit_check_runs_green_on_a_clean_public_page—tests/kit_checks_tests.rsa11y_layer_maps_axe_violations—tests/layer_tests.rscapture_layers_fold_console_and_network—tests/layer_tests.rsflaky_layer_classifies_alternating_history—tests/layer_tests.rsi18n_layer_flags_raw_key_leak—tests/layer_tests.rslayered_check_metrics_reach_the_run_report—tests/layer_tests.rsperf_layer_emits_metric_and_can_breach—tests/layer_tests.rspii_guard_fails_on_private_asset_servable_url_on_public_surface—tests/layer_tests.rspii_guard_passes_for_a_legitimately_public_asset—tests/layer_tests.rsstate_layer_rejects_500_where_empty_expected—tests/layer_tests.rsviewport_layer_attributes_failure—tests/layer_tests.rslive_public_smoke_check—tests/live_browser_tests.rsb016_inner_panics_after_creating_a_run_db—tests/run_db_tests.rscreate_database_statement_is_quoted—tests/run_db_tests.rsdrop_reclaims_the_database_when_the_test_panics—tests/run_db_tests.rslive_clone_from_template_then_drop—tests/run_db_tests.rsquote_identifier_doubles_embedded_quotes—tests/run_db_tests.rsrun_db_name_is_unique_and_valid—tests/run_db_tests.rsswap_database_in_url_replaces_only_the_db_segment—tests/run_db_tests.rsvalidate_identifier_accepts_safe_names—tests/run_db_tests.rs- _… 113 more_
Documentation coverage
| Measure | Documented | Total |
|---|---|---|
| Public items with rustdoc | 178 | 237 |
Public modules with a //! block | 27 | 27 |
pie showData
title Public items with rustdoc
"Documented" : 178
"No rustdoc detected" : 59
Metrics
| Metric | Value |
|---|---|
| Rust source files | 28 |
| Source lines | 7190 |
| Code lines | 4732 |
| Public API items | 237 |
| Public modules | 27 |
| Tests | 143 |
| Examples | 1 |
| Cargo features | 2 |
| Direct runtime dependencies | 15 |
| Workspace reverse dependencies | 0 |
pie showData
title Public API by kind
"constant" : 1
"enum" : 9
"function" : 41
"method" : 138
"struct" : 37
"trait" : 10
"type alias" : 1
pie showData
title Rust source composition
"Code" : 4732
"Blank or comment" : 2458
Generation
Rendered by tools-corpus corpus readme from repository evidence alone, renderer schema 2, lexicon current. No model, network service or database was consulted. Regenerate with tools-corpus corpus readme --write; verify with --check.