foundation-text-clustering

Local-only embedding-based text clustering: groups texts by vector similarity into opaque, zero-based cluster labels. The embedding vector never leaves this call -- only the cluster label crosses any further disclosure boundary, since a raw vector for a short/constrained phrase can often be inverted back toward the source text (unlike a one-way opaque ID). Generic over infrastructure_embeddings::Embedder -- works with the dependency-free DeterministicEmbedder (lexical-overlap only, not semantic -- the honest default, no model download required) or the real ONNX BGE-m3 embedder behind that crate's `ort` feature (genuinely semantic, opt-in, needs the model present).

What it is for

Local-only embedding-based text clustering: group texts by vector similarity into opaque, zero-based cluster labels via a simple greedy nearest-centroid pass (deterministic given a deterministic embedder and threshold -- no randomized initialization, no iteration count to tune).

The embedding vector never leaves this call. Only the cluster label crosses any further disclosure boundary. This matters: a raw embedding vector for a short, constrained phrase (a citation, a named entity) can often be inverted or nearest-neighbor-matched back toward the source text -- a real, published attack class. A one-way, non-invertible cluster label carries no such geometry to exploit, the same reason table-shard/transcript-shard/prose-shard disclose opaque P1/D1-style IDs and never a raw identifier.

Generic over infrastructure_embeddings::Embedder, so it works unchanged with either implementation already in this repo: the dependency-free DeterministicEmbedder (lexical/word-overlap similarity only -- NOT semantic, and this crate makes no claim otherwise; see that crate's own doc comment) with zero model download required, or the real ONNX BGE-m3 embedder behind infrastructure-embeddings's ort feature (genuinely semantic, opt-in, requires the model to actually be present).

Capabilities

crate root

Local-only embedding-based text clustering: group texts by vector

How to use it

No examples/ target and no doctest in this crate's rustdoc. The tests listed under Verification are the closest executable usage.

Module structure

No public modules: the crate root is its whole surface.

Public surface

`crate root`

No pub use re-exports: every item above is declared in this crate.

Boundary

Reaches into infrastructure.

Shares tier foundation with 27 other crates: foundation-audit-log, foundation-basemodels, foundation-bounded-io, foundation-conversation-closure, foundation-crypto-sign, foundation-decisioning, foundation-encounter-vocabulary, foundation-fs-metadata, … (27 total).

_What this crate deliberately does NOT own is a judgment. No committed registry records one for it, so none is stated here._

Where it sits

Tier flow. Which tiers this crate's own edges cross.

flowchart LR
  n_foundation["foundation"] --> n_infrastructure["infrastructure"]

Dependencies

Runtime, in this workspace.

Development, from outside the workspace.

Build. None.

Depended on by. 1 workspace crate.

Signal flow — what reaches this crate, and what it reaches.

flowchart LR
  n_domain_prose_shard["domain-prose-shard"] -->|uses| SELF
  SELF["foundation-text-clustering"]
  SELF -->|runtime| n_infrastructure_embeddings["infrastructure-embeddings"]
  classDef self fill:#1f883d,stroke:#1f883d,color:#fff;
  class SELF self;

Feature flags

No Cargo features are defined: every capability is unconditional, so no consumer can receive a half-wired crate.

Targets

Error model

No public error type was detected: no public item declares a type named *Error, and no public signature returns one.

Operational characteristics

No unsafe block, unsafe fn, unsafe impl or unsafe trait was found by the parser anywhere in this crate's source.

Configuration

No environment variable is read with a literal name anywhere in this crate. A variable whose key is computed at run time cannot be listed here, and is not claimed to be absent.

Related capabilities

1 workspace crate depends on this one: domain-prose-shard.

Verification

Evidence by module. How often each public module is named by something executable.

What the tests establish, by name:

• a_maximal_threshold_never_merges_non_identical_text — src/lib.rs
• a_single_text_gets_cluster_zero — src/lib.rs
• empty_input_returns_empty_labels — src/lib.rs
• heavily_overlapping_text_clusters_with_completely_different_text_separately — src/lib.rs
• identical_texts_always_cluster_together — src/lib.rs

Documentation coverage

pie showData
    title Public items with rustdoc
    "Documented" : 1
    "No rustdoc detected" : 0

Metrics

pie showData
    title Public API by kind
    "function" : 1

pie showData
    title Rust source composition
    "Code" : 103
    "Blank or comment" : 56

Generation

Rendered by tools-corpus corpus readme from repository evidence alone, renderer schema 2, lexicon current. No model, network service or database was consulted. Regenerate with tools-corpus corpus readme --write; verify with --check.