infrastructure-onnx

Generic, model-agnostic local ONNX inference runtime (named tensor in -> named tensor out) plus a GPU concurrency/model-residency limiter. Lifts the ort+CUDA session wiring so vision and text models share one device code path (ADR 0020). Default build is dependency-free (stub + limiter); the real ONNX Runtime model is behind feature `ort`.

What it is for

infrastructure-onnx — a generic, model-agnostic local ONNX inference runtime.

Lifts the ORT + CUDA session wiring out of infrastructure-embeddings so vision models (SCRFD/ArcFace/YOLO) and text models can share one device code path (ADR 0020). This crate is deliberately dumb: named f32 tensors in → named f32 tensors out. No tokenizer, no pooling, no dimension assumptions, no privilege policy — those belong to the consuming crate.

The default build is dependency-free: the InferenceModel trait, a StubModel for dev/tests, and the GpuLimiter / ModelResidency device-arbitration knobs (ADR 0020 §6) all build with zero native deps. The real in-process runtime (OnnxModel) lives behind the ort feature and drops in behind the same trait.

use infrastructure_onnx::{InferenceModel, NamedTensor, StubModel, Tensor};
let out = vecNamedTensor::new("y", Tensor::new(vec![1, 2, vec1.0, 2.0).unwrap())];
let model = StubModel::new("demo", out);
let y = model.run(vec).unwrap();
assert_eq!(y.len(), 1);

Capabilities

InferenceModel

infrastructure-onnx — a generic, model-agnostic local ONNX inference runtime.

How to use it

From this crate's own rustdoc:

use infrastructure_onnx::{InferenceModel, NamedTensor, StubModel, Tensor};
let out = vec![NamedTensor::new("y", Tensor::new(vec![1, 2], vec![1.0, 2.0]).unwrap())];
let model = StubModel::new("demo", out);
let y = model.run(vec![]).unwrap();
assert_eq!(y.len(), 1);

Module structure

No public modules: the crate root is its whole surface.

Public surface

`crate root`

Re-exports. Exported here, defined elsewhere.

Boundary

Depends on no other workspace tier.

Shares tier infrastructure with 82 other crates: infrastructure-acquire, infrastructure-adapters-google-calendar, infrastructure-adapters-google-gmail, infrastructure-adapters-google-places, infrastructure-adapters-google-trends, infrastructure-adapters-shodan, infrastructure-adapters-yelp, infrastructure-agent, … (82 total).

_What this crate deliberately does NOT own is a judgment. No committed registry records one for it, so none is stated here._

Where it sits

Dependencies

Runtime, from outside the workspace.

Development. None.

Build. None.

Depended on by. 1 workspace crate.

Signal flow — what reaches this crate, and what it reaches.

flowchart LR
  n_infrastructure_vision["infrastructure-vision"] -->|uses| SELF
  SELF["infrastructure-onnx"]
  classDef self fill:#1f883d,stroke:#1f883d,color:#fff;
  class SELF self;

Feature flags

flowchart LR
  n_default["default"]
  n_ort["ort"] --> n_dep_ort["dep:ort"]
  n_ort["ort"] --> n_dep_ndarray["dep:ndarray"]

Targets

Error model

No public error type was detected: no public item declares a type named *Error, and no public signature returns one.

Operational characteristics

No unsafe block, unsafe fn, unsafe impl or unsafe trait was found by the parser anywhere in this crate's source.

Configuration

No environment variable is read with a literal name anywhere in this crate. A variable whose key is computed at run time cannot be listed here, and is not claimed to be absent.

Related capabilities

1 workspace crate depends on this one: infrastructure-vision.

Verification

Evidence by module. How often each public module is named by something executable.

What the tests establish, by name:

• residency_evicts_lru_over_the_cap — src/limiter.rs
• residency_zero_is_unlimited — src/limiter.rs
• try_acquire_respects_the_concurrency_cap — src/limiter.rs
• zero_means_unlimited_but_still_counts — src/limiter.rs
• stub_echoes_outputs_and_counts_calls — src/stub.rs

Documentation coverage

pie showData
    title Public items with rustdoc
    "Documented" : 1
    "No rustdoc detected" : 0

Metrics

pie showData
    title Public API by kind
    "trait" : 1

pie showData
    title Rust source composition
    "Code" : 389
    "Blank or comment" : 201

Generation

Rendered by tools-corpus corpus readme from repository evidence alone, renderer schema 2, lexicon current. No model, network service or database was consulted. Regenerate with tools-corpus corpus readme --write; verify with --check.