AWS Service Bring-Up Process
This document describes the process for going from zero to full Alchemy coverage for a single AWS service:
- all canonical resources
- all bindings
- all event sources
- all ergonomic helpers
- deterministic audit and test coverage checks
Use this process whenever adding a brand new AWS service or finishing an incomplete one.
Goal
For a given AWS service, the end state should include:
- Every canonical Alchemy resource for that service.
- Every important AWS API operation represented either as:
- a binding,
- a resource lifecycle provider,
- an event source surface,
- or an intentional helper abstraction.
- Runtime-specific event-source implementations where applicable.
- End-to-end tests covering the implemented binding and event-source surface.
- Deterministic audit checks that report what is still missing.
Source Of Truth
Start from the distilled spec in:
submodules/distilled/packages/aws/src/services/<service>.ts
Never start from ad-hoc memory of the AWS service. The distilled spec is the source of truth for operations.
Core Concepts
Every distilled operation must be classified into one of these buckets:
1. Binding
Use a binding when the operation is a runtime capability.
Examples:
GetItem(table)PutItem(table)ListTables()DescribeTable(table)
Bindings are:
- one file per operation
- the combined
Binding.Serviceform (interface X extends Binding.Service<X, "id", Shape>+const X = Binding.Service<X>("id")); the deploy-time IAM registration is inlined into the impl layer underif (!globalThis.__ALCHEMY_RUNTIME__), resolving the host viayield* Binding.host - usually named
packages/aws/src/<Service>/<Operation>.ts(callable + types) with the impl layer in<Operation>Http.ts(AWS runtime impls call the distilled HTTP API authenticated by the Lambda's IAM role;Http, notBinding—Bindingis a Cloudflare native-worker concept)
2. Resource
Use a resource when the operation set implies lifecycle ownership of infrastructure.
Examples:
createTable/updateTable/deleteTable->TablecreateBucket/deleteBucket->Bucket
Resources are:
- canonical Alchemy infrastructure entities
- implemented as
Resourcecontract + provider in a single file
3. Event Source
Use an event source when the service can push records/events into a runtime.
This always has two layers:
- Service-level abstraction in
packages/aws/src/<Service>/... - Runtime-specific implementation in places like:
packages/aws/src/Lambda/...alchemy/src/Process/...
Examples:
consumeBucketEvents(bucket, handler)consumeQueueMessages(queue, handler)consumeTableChanges(table, handler)for DynamoDB-style change streams
4. Helper
Use a helper when multiple raw operations should collapse into a more ergonomic surface.
Examples:
consumeBucketEvents(bucket, handler)consumeQueueMessages(queue, handler)- batch or transaction wrappers
Helpers should not hide missing low-level primitives. Implement the primitives first.
Resource Arity
Classify each binding by resource arity:
0: service/account scoped- example:
ListTables
- example:
1: one resource- example:
GetItem(table)
- example:
2+: multiple resources- example:
RestoreTableToPointInTime(fromTable, toTable) - example: copy, batch, or transaction style operations
- example:
This classification helps decide:
- binding shape
- helper shape
- policy shape
- whether the operation belongs on a resource or service surface
Arity should be modeled in terms of canonical resources whenever possible.
- good: a
2-arity binding accepts<From extends Table, To extends Table> - bad: a
2-resource operation accepts oneTableplus a rawstringtarget name - only fall back to raw identifiers when there is no real canonical resource to bind against
- when there's a missing canonical resource, that might suggest we need to add one
Case Study: ExecuteTransaction
Use ExecuteTransaction as the reference pattern for bindings that touch 1..* canonical resources.
The ambiguity we want to avoid is:
- bad:
ExecuteTransaction()with IAMResource: ["*"] - bad:
ExecuteTransaction(tableNames: string[]) - bad: a SID like
AWS.DynamoDB.ExecuteTransaction(2 table(s))that hides which resources were bound
The required pattern is:
- good:
ExecuteTransaction(tableA, tableB, ...) - good: the binding type requires at least one table
- good: the policy enumerates exactly those table ARNs
- good: the SID is deterministic and names the participating resources
Runbook for any 1..* resource-bound binding:
- Model the binding arguments as a non-empty tuple of canonical resources.
- Call
.bind(resourceA, resourceB, ...), never.bind()with hidden resource discovery. - Before constructing the SID, sort the resources by
LogicalIdso equivalent calls produce the same binding identity. - Pass the sorted resource array into the
host.bindtemplate so the SID renders each resource name explicitly, for exampleAWS.DynamoDB.ExecuteTransaction(TableA, TableB). - Build IAM
Resourcefrom those same sorted resources, for examplesortedTables.map((table) => table.tableArn). - Only use
Resource: ["*"]if the operation is truly service-scoped or AWS IAM does not support resource-level scoping for that API.
Reference shape:
type ExecuteTransactionTables = [Table, ...Table[]];
const sortedTables = [...tables].sort((a, b) =>
a.LogicalId.localeCompare(b.LogicalId),
);
yield *
host.bind`Allow(${host}, AWS.DynamoDB.ExecuteTransaction(${sortedTables}))`({
policyStatements: [
{
Effect: "Allow",
Action: [
"dynamodb:PartiQLSelect",
"dynamodb:PartiQLInsert",
"dynamodb:PartiQLUpdate",
"dynamodb:PartiQLDelete",
],
Resource: sortedTables.map((table) => table.tableArn),
},
],
}); Full Bring-Up Loop
Follow this loop until audit is clean or only intentionally deferred items remain.
Step 1: Run Audit
Run:
pnpm audit:service dynamodb The audit should report:
- implemented bindings
- missing bindings
- resource lifecycle ops
- event source ops
- helper candidates
- registration gaps
- missing binding tests
Step 2: Build The Service Model
Before coding, explicitly answer:
- What are the canonical resources?
- What bindings belong to each resource?
- What service-scoped bindings exist?
- What event-source surfaces exist?
- What helpers should exist?
- Which items are intentionally deferred?
For DynamoDB, for example:
- canonical resource:
Table - folded table-owned surface: local/global secondary indexes live on
Table, not a standaloneSecondaryIndexresource - bindings: item/table/admin operations
- event source: Kinesis streaming destination / change stream surface
- helper:
consumeTableChanges(table, props?, handler)
Step 3: Implement Missing Bindings
Implement the smallest coherent slice first.
Good order:
- read/admin bindings
- write/update bindings
- transaction/batch bindings
- restore or special-case bindings
Binding conventions:
- one file per operation
- no auto-marshalling
- user passes raw AWS SDK/distilled types
- the binding should mostly inject resource identifiers like
TableName - policies should be explicit and minimal
- if an operation is
2+-arity, the binding should capture all participating resources so the policy can stay least-privilege - never use
Resource: ["*"]for a resource-bound binding if it can be avoided by passing canonical resources to.bind(...) - if an operation touches
1..*canonical resources, model the binding to accept those resources explicitly so the policy can enumerate only those ARNs Resource: ["*"]is only acceptable when the operation is truly service-scoped or AWS does not support narrower resource-level IAM for that API- do not add IAM
Sidfields in binding policy statements unless there is a demonstrated AWS requirement for one
Example:
- good:
GetItemRequest extends Omit<GetItemInput, "TableName"> - bad: replacing AWS input types with custom marshalled
Record<string, any>
Step 4: Register Everything
After each binding/resource implementation, update:
packages/aws/src/<Service>/index.tspackages/aws/src/Providers.ts
If registration is missing, audit should flag it.
Step 5: Add Binding Tests Immediately
Every implemented binding should have a corresponding describe("<BindingName>") block in:
packages/integration-tests/test/AWS/<Service>/Bindings.test.ts
Examples:
describe("GetItem", () => {
test("gets an existing item", ...);
test("returns undefined for missing item", ...);
}); describe("PutItem", () => {
test("puts an item into the table", ...);
}); Deterministic rule:
- if a binding exists, audit should warn if the matching
describe("<BindingName>")block is missing
Step 6: Use A Real Lambda Fixture
Use a real Lambda fixture in:
packages/integration-tests/test/AWS/<Service>/handler.ts
Pattern:
- create resource(s)
- bind operations
- expose HTTP endpoints for each tested operation
- use those endpoints from the E2E test
This keeps tests end-to-end while still giving fine-grained per-binding coverage.
Layer provisioning rule:
- when a Lambda fixture provides both composite layers and foundational binding layers, do not put them all in one flat
Layer.mergeAll(...) - composite layers include event sources, sinks, and higher-level helpers that themselves depend on lower-level bindings
- foundational layers include the binding/capability implementations such as
GetItemLive,PutObjectLive,PublishLive,PublishBatchLive, and similar - use
Effect.provide(Layer.provideMerge(...))so the foundational layer group is provided to the composite layer group - otherwise
Layer.mergeAll(...)only unions outputs and requirements, and sibling layers do not satisfy each other's requirements
Required shape:
Effect.provide(
Layer.provideMerge(
Layer
.mergeAll
// composite services: event sources, sinks, helpers
(),
Layer
.mergeAll
// foundational bindings/capabilities they depend on
(),
),
); Example failure mode:
TopicSinkLivedepends onPublishBatch- if
TopicSinkLiveandPublishBatchLiveare only siblings in the sameLayer.mergeAll(...), the final Lambda effect still requiresPublishBatch - grouping them with
Layer.provideMerge(...)removes that leaked requirement
Step 7: Make Setup Observable
Fixture setup must log clearly:
- destroying previous resources
- deploying fixture
- function URL
- readiness probe URL
- readiness retries
- readiness success
If setup is confusing, add logs before adding retries.
Step 8: Keep Readiness Failure Fast
Do not wait minutes for a function to become ready.
Current convention:
- if the function is not ready in about 20 seconds, fail the setup
Use a short retry budget and log each failed readiness attempt.
Step 9: Implement Missing Resource Surface
Once bindings are in good shape, fill missing resource-level gaps:
- missing resource providers
- placeholder resources
- resource update/delete gaps
- missing nested infrastructure surfaces such as indexes or replicas
Important:
- audit may only recognize canonical resources implied directly by lifecycle operations
- you must still inspect the service for real resource-shaped gaps not fully inferred by the script
Example:
- a placeholder like
SecondaryIndex.tsstill counts as incomplete coverage until it is either removed or folded into the canonical resource model
Step 10: Implement Event Sources
For stream/notification services:
- define a service-level abstraction
- add runtime-specific implementation(s)
- add helper surface
- add tests
When the event source needs an intermediate canonical resource, the binding should create that resource automatically instead of forcing user code to instantiate it. SNS is the reference case:
- the public binding is
notifications(topic, handler) - the Lambda runtime policy creates the
Subscriptionresource automatically - any service-to-Lambda invoke permission stays in the runtime policy layer that wires the event source, not in user code
- the canonical resource still exists and can be used directly when needed; the helper just creates it on behalf of the user
For DynamoDB-style changes this likely means:
- table-side stream/destination surface
- Lambda runtime integration
consumeTableChanges(table, props?, handler)helper- E2E coverage
Step 11: Implement Helpers
After low-level primitives exist, add ergonomic helpers for:
- stream subscriptions
- batch operations
- transactions
Helpers should feel native to Alchemy and match established service patterns.
Case Study: DynamoDB Streams
DynamoDB Streams is the reference pattern for mutable event-source configuration that belongs to a canonical resource but still needs binding-based composition.
Required shape:
- the canonical resource remains
Table Tablekeeps stream state in its attributes, but does not acceptstreamSpecificationas a plain input prop- stream enablement is requested through the table binding contract
- the public helper is
consumeTableChanges(table, props?, handler) - the service-level abstraction lives in
packages/aws/src/DynamoDB/Stream.ts - the Lambda runtime implementation lives in
packages/aws/src/Lambda/TableEventSource.ts
Why this pattern exists:
- stream enablement mutates the table itself
- the consumer is another resource, usually a Lambda Function
- prop-driven stream configuration makes circular composition awkward
- bindings let the consumer request the mutation while
Tablestays the canonical owner of stream state
Implementation rules:
- The helper attaches stream requirements to
Tablethrough bindings. - The
Tableprovider derives the effective stream configuration from bindings duringcreateandupdate. - Zero stream bindings means the table stream should be disabled.
- Multiple bindings may coexist only when they request the same
StreamViewType. - Conflicting
StreamViewTyperequests must fail deterministically before AWS calls are made. - Runtime-specific layers handle IAM, host wiring, and event-source mapping resources; they do not move stream ownership out of
Table.
Lambda-first slice:
- implement
consumeTableChanges(table, props?, handler)first for Lambda - the Lambda layer binds the table stream requirement, grants stream-read IAM, and creates
AWS.Lambda.EventSourceMapping - Process or other runtimes can be added later without changing
Tableback to a prop-driven stream model
Step 12: Re-Run Tests And Audit
After every meaningful slice:
- run the service-specific E2E tests
- rerun
pnpm audit:service <service> - use the updated output to choose the next slice
Repeat until:
- no important missing bindings remain
- resource surface is complete
- event-source surface is complete
- tests are in place
Deterministic Checks We Want
The audit should help enforce:
- Missing bindings from distilled operations.
- Missing registration in
index.ts. - Missing registration in
Providers.ts. - Missing
describe("<BindingName>")blocks inBindings.test.ts. - Avoidable
Resource: ["*"]usage in non-zero-arity bindings.
Over time it should also grow to flag:
- placeholder resources with no provider
- missing event-source surface for detected stream operations
- missing helper surface for known event-source patterns
AWS-Specific Conventions Learned
No Auto-Marshalling
Bindings do not auto-marshall request/response payloads for DynamoDB-style operations.
User responsibility:
- pass raw AWS/distilled input types
- marshal/unmarshal attribute values themselves
Output.interpolate
Only use Output.interpolate when actually composing a string.
Examples:
- good:
table.tableArn - good:
Output.interpolate\${table.tableArn}/index/*`` - bad:
Output.interpolate\${table.tableArn}``
Effect.orDie
For Lambda test fixtures, apply Effect.orDie once at the outer request handler layer, not repeatedly per route.
Binding Test Structure
Use one describe("<BindingName>") block per binding.
Inside that block:
- happy path
- relevant unhappy paths
Do not lump all bindings into one large undifferentiated test block.