Module connector_s3 

Amazon S3 connector.

Lists and downloads objects from an S3 bucket using the S3 REST API with AWS Signature V4 authentication. Implements pagination for large buckets, glob-based filtering on object keys, and supports custom endpoints for S3-compatible services (MinIO, LocalStack).

Uses only pure-Rust dependencies (hmac, sha2) for AWS signing — no C library dependencies like aws-lc-sys, making it compatible with all build environments including Nix.

Configuration

[connectors.s3.runbooks]
bucket = "acme-docs"
prefix = "engineering/runbooks/"
region = "us-east-1"
include_globs = ["**/*.md"]
# endpoint_url = "http://localhost:9000"   # MinIO

Environment Variables

Credentials are read from environment variables:

• AWS_ACCESS_KEY_ID — required
• AWS_SECRET_ACCESS_KEY — required
• AWS_SESSION_TOKEN — optional (for temporary credentials / IAM roles)

Authentication

All S3 requests are signed using AWS Signature Version 4. The signing implementation uses HMAC-SHA256 (hmac + sha2 crates).

Pagination

Large buckets (1000+ objects) are handled automatically via the ListObjectsV2 continuation token mechanism.

Content Type Detection

File extensions are mapped to MIME types:

Extension	MIME Type
.md	text/markdown
.txt	text/plain
.json	application/json
.yaml, .yml	text/yaml
.rst	text/x-rst
.html, .htm	text/html
Other	text/plain

Structs

AwsCredentials 🔒

AWS credentials loaded from environment variables.

S3Connector

An S3 connector instance that implements the Connector trait.

S3Object 🔒

Metadata for a single S3 object, parsed from ListObjectsV2 XML response.

Functions

build_globset 🔒

Build a [GlobSet] from a list of glob pattern strings.

derive_signing_key 🔒

Derive the AWS SigV4 signing key for a given date, region, and service.

detect_content_type 🔒

Detect MIME content type from a file extension.

download_object 🔒

Download a single object’s content from S3 using a signed GET request.

extract_xml_value 🔒

Extract the text content of an XML tag (simple, non-nested).

hex_hmac_sha256 🔒

Compute hex-encoded HMAC-SHA256.

hex_sha256 🔒

Compute the hex-encoded SHA-256 hash of data.

hmac_sha256 🔒

Compute HMAC-SHA256 of data with the given key.

list_objects 🔒

List all objects in the configured S3 bucket, handling pagination.

parse_list_objects_response 🔒

Parse a ListObjectsV2 XML response into a list of S3Objects.

s3_host 🔒

Compute the S3 hostname for the configured bucket and region.

scan_s3

Scan an S3 bucket and produce SourceItems.

uri_encode 🔒

URI-encode a string per RFC 3986 (used in SigV4 canonical requests).

Type Aliases

HmacSha256 🔒