data tier

data-import

Bulk data import for Rust applications

Bulk data import for Rust applications

Tierdata
Roleunclassified (baselined)
Pathcrates/data/import
Edition2021
Targetsdata_import
Public items68 across 0 modules
Tests36

What it is for

Bulk data import library crates for Rust applications.

Provides structured data import from various formats with validation and progress tracking.

Capabilities

ColumnMapping

Bulk data import library crates for Rust applications.

Item
pub struct ColumnMapping
ColumnMapping :: fn new(source : impl Into <String>, target : impl Into <String>) -> Self
ColumnMapping :: fn required(mut self) -> Self
ColumnMapping :: fn with_default(mut self, value : impl Into <String>) -> Self
ColumnMapping :: fn with_transformer(mut self, transformer : impl Into <String>) -> Self

CsvParser

Bulk data import library crates for Rust applications.

Item
pub struct CsvParser
CsvParser :: fn new(config : ImportConfig) -> Self
CsvParser :: fn parse(& self, content : & str) -> Result <Vec <HashMap <String, String>>>

DuplicateStrategy

Bulk data import library crates for Rust applications.

Item
pub enum DuplicateStrategy

ImportConfig

Bulk data import library crates for Rust applications.

Item
pub struct ImportConfig
ImportConfig :: fn default() -> Self
ImportConfig :: fn new(format : ImportFormat) -> Self
ImportConfig :: fn with_mapping(mut self, mapping : ColumnMapping) -> Self
ImportConfig :: fn with_mappings(mut self, mappings : Vec <ColumnMapping>) -> Self
ImportConfig :: fn with_validation(mut self, rule : ValidationRule) -> Self
ImportConfig :: fn with_delimiter(mut self, delimiter : char) -> Self
ImportConfig :: fn with_max_errors(mut self, max : usize) -> Self
ImportConfig :: fn with_batch_size(mut self, size : usize) -> Self
ImportConfig :: fn with_duplicate_strategy(mut self, strategy : DuplicateStrategy) -> Self
ImportConfig :: fn dry_run(mut self) -> Self
ImportConfig :: fn with_source_label(mut self, label : impl Into <String>) -> Self

ImportError

Bulk data import library crates for Rust applications.

Item
pub enum ImportError

ImportFormat

Bulk data import library crates for Rust applications.

Item
pub enum ImportFormat
ImportFormat :: fn extensions(& self) -> & & 'static str
ImportFormat :: fn from_extension(ext : & str) -> Option <Self>

ImportHandler

Bulk data import library crates for Rust applications.

Item
pub trait ImportHandler<T>

ImportJob:ImportJob

Bulk data import library crates for Rust applications.

Item
pub struct ImportJob

ImportJob:cancel

Bulk data import library crates for Rust applications.

Item
ImportJob :: fn cancel(& mut self) -> Result <()>

ImportJob:complete

Bulk data import library crates for Rust applications.

Item
ImportJob :: fn complete(& mut self)

ImportJob:fail

Bulk data import library crates for Rust applications.

Item
ImportJob :: fn fail(& mut self, error : impl Into <String>)

ImportJob:new

Bulk data import library crates for Rust applications.

Item
ImportJob :: fn new(name : impl Into <String>, config : ImportConfig) -> Self

ImportJob:record

Bulk data import library crates for Rust applications.

Item
ImportJob :: fn record_success(& mut self)
ImportJob :: fn record_skipped(& mut self)
ImportJob :: fn record_error(& mut self, error : ImportRowError)

ImportJob:should

Bulk data import library crates for Rust applications.

Item
ImportJob :: fn should_abort(& self) -> bool

ImportJob:start

Bulk data import library crates for Rust applications.

Item
ImportJob :: fn start_validation(& mut self) -> Result <()>
ImportJob :: fn start_processing(& mut self, total_rows : usize)

ImportJob:statistics

Bulk data import library crates for Rust applications.

Item
ImportJob :: fn statistics(& self) -> ImportStatistics

ImportJob:update

Bulk data import library crates for Rust applications.

Item
ImportJob :: fn update_progress(& mut self, processed : usize)

ImportJob:with

Bulk data import library crates for Rust applications.

Item
ImportJob :: fn with_file(mut self, name : impl Into <String>, size : u64) -> Self
ImportJob :: fn with_created_by(mut self, user_id : Uuid) -> Self

ImportRowError

Bulk data import library crates for Rust applications.

Item
pub struct ImportRowError
ImportRowError :: fn new(row : usize, message : impl Into <String>) -> Self
ImportRowError :: fn with_column(mut self, column : impl Into <String>) -> Self
ImportRowError :: fn with_value(mut self, value : impl Into <String>) -> Self

ImportStatistics

Bulk data import library crates for Rust applications.

Item
pub struct ImportStatistics

ImportStatus

Bulk data import library crates for Rust applications.

Item
pub enum ImportStatus
ImportStatus :: fn is_terminal(& self) -> bool
ImportStatus :: fn can_cancel(& self) -> bool

ImportTemplate

Bulk data import library crates for Rust applications.

Item
pub struct ImportTemplate
ImportTemplate :: fn new(name : impl Into <String>, entity_type : impl Into <String>, config : ImportConfig,) -> Self
ImportTemplate :: fn with_description(mut self, description : impl Into <String>) -> Self
ImportTemplate :: fn with_sample_file(mut self, path : impl Into <String>) -> Self
ImportTemplate :: fn create_job(& self, name : impl Into <String>) -> ImportJob

KeywordCsvVendor

Bulk data import library crates for Rust applications.

Item
pub enum KeywordCsvVendor
KeywordCsvVendor :: fn as_str(& self) -> & 'static str
KeywordCsvVendor :: fn detect(headers : & String) -> Self
KeywordCsvVendor :: fn resolve_mappings(& self, headers : & String) -> Vec <ColumnMapping>
KeywordCsvVendor :: fn detect_columns(headers : & String) ->(Self, Vec <ColumnMapping>)

Result

Bulk data import library crates for Rust applications.

Item
pub type Result<T>: std::result::Result <T, ImportError>

ValidationRule

Bulk data import library crates for Rust applications.

Item
pub struct ValidationRule
ValidationRule :: fn required(field : impl Into <String>) -> Self
ValidationRule :: fn max_length(field : impl Into <String>, max : usize) -> Self
ValidationRule :: fn email(field : impl Into <String>) -> Self
ValidationRule :: fn pattern(field : impl Into <String>, regex : impl Into <String>) -> Self
ValidationRule :: fn in_values(field : impl Into <String>, values : Vec <String>) -> Self
ValidationRule :: fn with_message(mut self, message : impl Into <String>) -> Self

ValidationType

Bulk data import library crates for Rust applications.

Item
pub enum ValidationType

How to use it

No examples/ target and no doctest in this crate's rustdoc. The tests listed under Verification are the closest executable usage.

Module structure

No public modules: the crate root is its whole surface.

Public surface

`crate root`

ItemWhat it is
pub enum ImportErrorImport errors.
pub type Result<T>: std::result::Result <T, ImportError>Result type for import operations.
pub enum ImportFormatSupported import formats.
ImportFormat :: fn extensions(& self) -> & & 'static strGet the expected file extensions for this format.
ImportFormat :: fn from_extension(ext : & str) -> Option <Self>Detect format from file extension.
pub enum ImportStatusImport job status.
ImportStatus :: fn is_terminal(& self) -> boolCheck if the import is in a terminal state.
ImportStatus :: fn can_cancel(& self) -> boolCheck if the import can be cancelled.
pub struct ColumnMappingColumn mapping for import.
ColumnMapping :: fn new(source : impl Into <String>, target : impl Into <String>) -> SelfCreate a new column mapping.
ColumnMapping :: fn required(mut self) -> SelfMark as required.
ColumnMapping :: fn with_default(mut self, value : impl Into <String>) -> SelfSet a default value.
ColumnMapping :: fn with_transformer(mut self, transformer : impl Into <String>) -> SelfSet a transformer.
pub struct ValidationRuleValidation rule for import data.
pub enum ValidationTypeTypes of validation rules.
ValidationRule :: fn required(field : impl Into <String>) -> SelfCreate a required field rule.
ValidationRule :: fn max_length(field : impl Into <String>, max : usize) -> SelfCreate a max length rule.
ValidationRule :: fn email(field : impl Into <String>) -> SelfCreate an email validation rule.
ValidationRule :: fn pattern(field : impl Into <String>, regex : impl Into <String>) -> SelfCreate a pattern validation rule.
ValidationRule :: fn in_values(field : impl Into <String>, values : Vec <String>) -> SelfCreate an allowed values rule.
ValidationRule :: fn with_message(mut self, message : impl Into <String>) -> SelfSet a custom error message.
pub struct ImportConfigImport configuration.
pub enum DuplicateStrategyStrategy for handling duplicate records.
ImportConfig :: fn default() -> Self
ImportConfig :: fn new(format : ImportFormat) -> SelfCreate a new import configuration.
ImportConfig :: fn with_mapping(mut self, mapping : ColumnMapping) -> SelfAdd a column mapping.
ImportConfig :: fn with_mappings(mut self, mappings : Vec <ColumnMapping>) -> SelfAdd multiple mappings.
ImportConfig :: fn with_validation(mut self, rule : ValidationRule) -> SelfAdd a validation rule.
ImportConfig :: fn with_delimiter(mut self, delimiter : char) -> SelfSet the delimiter (for CSV).
ImportConfig :: fn with_max_errors(mut self, max : usize) -> SelfSet the maximum number of errors before aborting.
ImportConfig :: fn with_batch_size(mut self, size : usize) -> SelfSet the batch size for processing.
ImportConfig :: fn with_duplicate_strategy(mut self, strategy : DuplicateStrategy) -> SelfSet the duplicate handling strategy.
ImportConfig :: fn dry_run(mut self) -> SelfEnable dry run mode.
ImportConfig :: fn with_source_label(mut self, label : impl Into <String>) -> SelfTag this config with a source-system label (e.g
pub struct ImportRowErrorA single import error record.
ImportRowError :: fn new(row : usize, message : impl Into <String>) -> SelfCreate a new row error.
ImportRowError :: fn with_column(mut self, column : impl Into <String>) -> SelfSet the column.
ImportRowError :: fn with_value(mut self, value : impl Into <String>) -> SelfSet the value.
pub struct ImportJobImport job representing an async import operation.
ImportJob :: fn new(name : impl Into <String>, config : ImportConfig) -> SelfCreate a new import job.
ImportJob :: fn with_file(mut self, name : impl Into <String>, size : u64) -> SelfSet the file information.
ImportJob :: fn with_created_by(mut self, user_id : Uuid) -> SelfSet the creator.
ImportJob :: fn start_validation(& mut self) -> Result <()>Start the validation phase.
ImportJob :: fn start_processing(& mut self, total_rows : usize)Start the processing phase.
ImportJob :: fn update_progress(& mut self, processed : usize)Update progress.
ImportJob :: fn record_success(& mut self)Record a successful import.
ImportJob :: fn record_skipped(& mut self)Record a skipped row.
ImportJob :: fn record_error(& mut self, error : ImportRowError)Record an error.
ImportJob :: fn should_abort(& self) -> boolCheck if the import should abort due to too many errors.
ImportJob :: fn complete(& mut self)Mark the import as completed.
ImportJob :: fn fail(& mut self, error : impl Into <String>)Mark the import as failed.
ImportJob :: fn cancel(& mut self) -> Result <()>Cancel the import.
ImportJob :: fn statistics(& self) -> ImportStatisticsGet import statistics.
pub struct ImportStatisticsImport statistics summary.
pub struct ImportTemplateImport template for reusable import configurations.
ImportTemplate :: fn new(name : impl Into <String>, entity_type : impl Into <String>, config : ImportConfig,) -> SelfCreate a new import template.
ImportTemplate :: fn with_description(mut self, description : impl Into <String>) -> SelfSet description.
ImportTemplate :: fn with_sample_file(mut self, path : impl Into <String>) -> SelfSet sample file path.
ImportTemplate :: fn create_job(& self, name : impl Into <String>) -> ImportJobCreate an import job from this template.
pub trait ImportHandler<T>Trait for import record handlers.
pub struct CsvParserSimple CSV parser for synchronous imports.
CsvParser :: fn new(config : ImportConfig) -> SelfCreate a new CSV parser.
CsvParser :: fn parse(& self, content : & str) -> Result <Vec <HashMap <String, String>>>Parse CSV content into rows.
pub enum KeywordCsvVendorKnown keyword-research CSV export vendors, plus a generic fallback
KeywordCsvVendor :: fn as_str(& self) -> & 'static strLowercase label for this vendor (matches this crate's serde snake_case wire format), suitable for ImportConfig::with_source_label.
KeywordCsvVendor :: fn detect(headers : & String) -> SelfDetect which vendor's known header spellings a real CSV header row matches, by looking for each vendor's distinctive markers -- header spellings not shared with any other vendor's candidate list for the same field
KeywordCsvVendor :: fn resolve_mappings(& self, headers : & String) -> Vec <ColumnMapping>Resolve this vendor's known header-name spellings against a real CSV header row
KeywordCsvVendor :: fn detect_columns(headers : & String) ->(Self, Vec <ColumnMapping>)Single entry point: detect the vendor from the header row, then resolve its column mappings against those same headers

No pub use re-exports: every item above is declared in this crate.

Boundary

Depends on no other workspace tier.

Shares tier data with 2 other crates: data-export, data-webhooks.

_What this crate deliberately does NOT own is a judgment. No committed registry records one for it, so none is stated here._

Where it sits

Tier (ontology)data
Architectural role (taxonomy)unclassified (baselined)
Locationcrates/data/import
Vocabulary in force (lexicon)current

Dependencies

Runtime, from outside the workspace.

CrateRequirementFeaturesOptionalOnly on
async-trait^0.1noalways
chrono^0.4serdenoalways
serde^1derivenoalways
serde_json^1noalways
thiserror^2noalways
uuid^1v4, v7, serde, jsnoalways

Development, from outside the workspace.

CrateRequirementFeaturesOptionalOnly on
tokio^1fullnoalways

Build. None.

Depended on by. 1 workspace crate.

Signal flow — what reaches this crate, and what it reaches.

flowchart LR
  n_application_import["application-import"] -->|uses| SELF
  SELF["data-import"]
  classDef self fill:#1f883d,stroke:#1f883d,color:#fff;
  class SELF self;

Feature flags

No Cargo features are defined: every capability is unconditional, so no consumer can receive a half-wired crate.

Targets

KindNameSource
libdata_import`src/lib.rs`

Error model

Error typeNamed by
ImportErrorResult
ImportRowErrordeclared, no public signature returns it

Operational characteristics

PropertyEvidence
async public surfacenone detected
async runtimenone detected
database accessnone detected
network I/Onone detected
unsafe codenone detected
environment variablesnone detected

No unsafe block, unsafe fn, unsafe impl or unsafe trait was found by the parser anywhere in this crate's source.

Configuration

No environment variable is read with a literal name anywhere in this crate. A variable whose key is computed at run time cannot be listed here, and is not claimed to be absent.

1 workspace crate depends on this one: application-import.

Verification

KindCount
Unit tests36
Integration tests0
Examples0
Doctests0

Evidence by module. How often each public module is named by something executable.

ModuleTestsExamplesConsumers
crate root1601

What the tests establish, by name:

Documentation coverage

MeasureDocumentedTotal
Public items with rustdoc6768
Public modules with a //! block00
pie showData
    title Public items with rustdoc
    "Documented" : 67
    "No rustdoc detected" : 1

Metrics

MetricValue
Rust source files1
Source lines1442
Code lines1144
Public API items68
Public modules0
Tests36
Examples0
Cargo features0
Direct runtime dependencies6
Workspace reverse dependencies1
pie showData
    title Public API by kind
    "enum" : 6
    "method" : 52
    "struct" : 8
    "trait" : 1
    "type alias" : 1
pie showData
    title Rust source composition
    "Code" : 1144
    "Blank or comment" : 298

Generation

Rendered by tools-corpus corpus readme from repository evidence alone, renderer schema 2, lexicon current. No model, network service or database was consulted. Regenerate with tools-corpus corpus readme --write; verify with --check.

All data · Manual