sailfork / t3code-github / .repos/ alchemy-effect/ .agents/ skills/ alchemy-provider/ review.md

Provider review gate

Each row turns a rule from AGENTS.md or the skill into a check with a command and an expected result. The rows come from the first review of the GCP provider and the corrections made during it; on the pre-fix GCP code these greps returned hundreds of hits each.

Run it at every wave boundary and before calling a provider done. Use a fresh reviewer (a subagent with no stake in the code) and have it report, for every item, the command it ran, the count, and the file:line of each hit it did not accept. "Looks fine" is not an answer. Every accepted hit needs a one-line reason.

<P> is the provider name. Paths are relative to the repo root: src/<P> means packages/<p>/src (the provider package) and test/<P> means packages/integration-tests/test/<P>. The greps find candidates; read each hit.

0. Coverage

Run this first; the other sections only judge the code that exists.

CheckCommandExpect
Every service catalogedcompare ls submodules/distilled/packages/<pkg>/src/services with ls processes/<P>/catalogone catalog file per service
Nothing left unbuiltcount statuses in processes/<P>/catalog/INDEX.mdno missing or implemented rows; every out-of-scope and blocked row has a reason or exact error
Index matches the codeevery tested row has a Resource<...>("<P>.<Service>.<Name>") in src/<P> and a registered provider in Providers.tsyes
One test file per resourceevery tested row has test/<P>/<Service>/<Name>.test.tsyes
Operations classifiedevery operation in each distilled service module appears in its catalog file as resource-lifecycle, binding, event-source, or internal with a reasonyes; the provider's port of scripts/audit-service.ts prints no unclassified operations
Binding coveragethe audit's missingBindings per service, and rg -l 'Binding\.Service<' src/<P> | xargs -n1 dirname | sort -u | wc -l against the services with binding operationsno missing bindings; every service with data-plane operations has bindings (AWS: 205 of 212)
Bindings through distilledread each *Http.tswraps a @distilled.cloud/<pkg> operation; no HttpClientRequest
Non-resource deliverablesindex rows for runtime, event sources, websites, examplesnone missing
Skill freshnessgit log -1 --format=%H -- .agents/skills/alchemy-provider vs the Skill: <sha> line at the top of INDEX.mdequal; a newer skill commit means re-read it and add rows for its new deliverables
Framework examplesls examples | grep '^<p>-website-' vs ls examples | grep '^hetzner-website-'one per framework the provider hosts (skip when it can't host websites)
Workload examplesls examples | grep '^<p>-' against the workload table in Step 10one per workload the cloud supports; each registered in tsconfig.json and scripts/test-examples.ts, with a passing test/integ.test.ts
Docs generated for everythingls website/src/content/docs/providers/<P> after pnpm docs:genone page per resource and binding page group, matching the index
Out-of-scope list publishedthe overview page's "Out of scope" sectionmatches the out-of-scope rows

How to compare against what good looks like

For each service, open the closest AWS or Cloudflare equivalent and compare side by side: props naming, diff, reconcile, binding shape, test shape, JSDoc examples. Where the new provider differs, the difference needs a reason the platform forces. References:

ConcernReference
Small reconcile with labels, list, adoptionpackages/hetzner/src/SshKey.ts
Multi-aspect reconcilepackages/aws/src/DynamoDB/Table.ts, packages/aws/src/Kinesis/Stream.ts
Read/Write/ReadWrite bindings, shared scaffolding unexportedpackages/cloudflare/src/R2/
Bindings that reach any hostpackages/aws/src/AWSBindings.ts, packages/gcp/src/GCPBindings.ts, packages/aws/src/SQS/BindingHttp.ts
Package shapepackages/hetzner/ (package.json, tsconfig.json, src/Contributions.ts)
Bindings tested inside a deployed hostpackages/integration-tests/test/AWS/DynamoDB/Bindings.test.ts + handler.ts
Framework websites on a CDN + functionspackages/aws/src/Website/ (S3 + CloudFront + Lambda), packages/cloudflare/src/Website/
Framework websites on containers/VMs (only when the cloud has no CDN or functions)packages/fly/src/Website/, packages/hetzner/src/Website/

1. Package

CheckCommandExpect
Exports generatedpnpm exports:checkup to date; no hand-written entries
Provider layerspnpm lint:providersno unknown/any requirement on providers()
Core untouchedgit diff origin/main --stat -- packages/alchemy/srcempty; anything core needs is a Contributions field
Package boundaryrg -n "from \"(@alchemy\.run/[\w-]+|alchemy)/.*internal" src/<P> and rg -n "\.\./\.\./[a-z-]+/src" src/<P>0; other packages reached only through their public exports
Contributionsread src/<P>/Contributions.ts and package.json alchemy.contributeslazy import() fields only; listed in contributes
Globalsls src/<P>/globals.d.ts and rg -n 'packages/<p>/src/globals' .gitignoreboth present
Registrationgit grep -il hetzner -- ':!packages/hetzner' ':!submodules' ':!examples' vs the same for <p>every file that lists Hetzner lists <P> where it applies
Versiongrep '"version"' packages/<p>/package.json packages/aws/package.jsonequal (packages release in lockstep)

2. Effect code

CheckCommandExpect
Nested yields (f(yield* x))rg -n 'yield\* [\w.]+\([^)]*yield\*' src/<P> test/<P> examples/<p>-*0; bind to a named const first
orDie in lifecycle or binding initrg -n 'orDie' src/<P>only in credential and layer wiring (Credentials.ts, Providers.ts)
Effect.promise / swallowed causesrg -n 'Effect\.promise|catchCause|catchAllCause' src/<P>0; use Effect.tryPromise with a typed error
Environment reads in resource coderg -n 'process\.env' src/<P>only AuthProvider.ts CI credentials
Widening castsrg -n 'as unknown as|as any' src/<P>0, or a comment naming the compiler limit

3. Errors

CheckCommandExpect
Forbidden treated as missingrg -n '"Forbidden"' src/<P> test/<P>never in a not-found/gone path; a 403-for-missing API gets its own distilled tag
Message string matchingrg -n 'message.*(includes|match|test)\(|/[^/]*not found[^/]*/i' src/<P>0; patch distilled with a matcher
Hand-written HTTPrg -n 'HttpClientRequest\.(get|post|put|delete|head|patch)\(' src/<P>every hit is a presigned/SAS URL from the API or the user's own deployed endpoint, with a comment saying so; data-plane and credential calls go through distilled
Catch-all errors handledrg -n 'Unknown\w*Error|HttpError' src/<P>0
Lists swallowing errorsrg -n 'orElseSucceed|catch\w*\(.*\[\]' src/<P>0 on any list/read path
Single-page listsread every list and findBy* helperpaginated (.items(...) or a page-token loop)

4. Lifecycle

CheckHowExpect
One operation waiterrg -c 'waitFor\w*Operation|pollOperation' src/<P>one definition in a shared module, many call sites
Realistic wait budgetsread each budget against the provider's documented create timeminutes for databases, clusters, environments; never a test-style 10 × 5s
Ready before returnread each reconcilereturns after the API reports ready, not after the create call
SQL migrationsfor each resource hosting a SQL schema, rg -n 'runMigrations' src/<P>a migrations prop driven by @alchemy.run/sql/Migrations (see SQL database migrations in AGENTS.md), with a live test that adds a file and redeploys
Identity change replacesfor each user-chosen name/id prop, read diffreplace; a rename test proves the old one is deleted
Ownership markersrg -n 'alchemy' src/<P> | rg -v 'label|tag|import'markers only in labels/tags, never in user-visible fields
Copied helpersrg -c 'const (isNotFound|isMissing|ignoreMissing|jsonEqual)' src/<P>one shared copy each
output === undefined branchesrg -n 'output === undefined' src/<P>none inside reconcile

5. Bindings and runtimes

CheckHowExpect
Bind, not host.bindrg -n 'host\.bind\(' src/<P> and rg -ln 'Bindings\.Bind' src/<P>bindings call <P>Bindings.Bind; no direct host.bind and no allowlist of host types
Foreign hostsread <P>Bindings.tsa non-native host gets a created identity + key bound into its env; withCredentials reads it at runtime
Other providers' typesrg -n '"(AWS|Cloudflare|GCP|Azure|Fly|Hetzner|Railway)\.[A-Z]' src/<P> minus <P>'s own prefix0; host access goes through RuntimeContext and core host services
Deploy-only readsrg -n 'Environment\.current' src/<P> in binding filesevery hit inside the !globalThis.__ALCHEMY_RUNTIME__ block
Native bundles foldrg -ln 'HostBundle' src/<P>applied by every host bundler the provider owns
External-host suitels test/<P>/ExternalHosta Cloudflare Worker (or other foreign host) calls the provider's bindings
Least privilegelist every role/permission each binding grants (rg -n 'roles/|permission' src/<P>)scoped to the bound resource; no admin/editor/owner unless the JSDoc says why
Revocationread the host's grant syncrevokes only grants Alchemy added; pre-existing grants survive
Secretsread each binding that hands out a credentialRedacted, never a plain env var or log line
Argument shapestype testcallables accept the resource, the Effect producing it, and the resolved value (WriteTable(table) and WriteTable(yield* table))
Runtimes identifiedrg -n 'runtime' processes/<P>/catalog/INDEX.md vs every resource that runs user codeeach one marked
Both formsfor each runtime: Platform( in its file, and an ### Async JSDoc sectioneffectful class and async main form both ship and are documented
InferEnvrg -n 'export type InferEnv' src/<P>exported, and used in every async example
Both forms testedls test/<P>/**/fixturesan effectful and an async fixture per runtime, each deployed and called
Workflow enginethe catalog's engine row, and rg -ln 'WorkflowEngine' src/<P>the cloud's durable engine has an Effect engine Layer passing the eight conformance cases from AGENTS.md, or the row is out-of-scope with the reason
Runtime coloringread each runtime callablerequires RuntimeContext; no WorkerEnvironment-style leaks
Event sourcescompare with AWS SQS.QueueEventSource / Cloudflare croncontract in the source service, per-host implementation layers
User-facing configread setup docs and examplesregion/account from profile or a <P>.Region("...") layer, matching AWS.Region; no new env vars

6. Websites

Skip this section when the provider can't host websites; then src/<P>/Website doesn't exist.

CheckCommandExpect
Hosting choicethe catalog's website entrynames the cloud's static hosting, CDN, and function runtime, with a reason for each
No containers by defaultrg -n 'ContainerApp|Run\.Service|ECS|Dockerfile|image' src/<P>/Website0, or the framework's JSDoc says why it needs a long-lived process and the cloud has no alternative
Static is compute-freeread StaticSite.ts and every assets-only path (Vite, SPA output)uploads to object/asset storage; no server, function, or VM
CDN by defaultread FrameworkSite.tsthe CDN is created without opting in; deploys purge or invalidate changed paths
Function targetls packages/frontend-frameworks/src/*/<p>.tsone per framework when SSR runs on functions; the Node target only on container/VM-only clouds
Live proofeach Website/<Framework>.test.tsasserts the response came through the CDN and an SSR route ran on the function

7. Tests

CheckCommandExpect
Gating flagsrg -oh 'process\.env\.\w+' test/<P> | sort | uniq -cone provider-wide slow flag, entitlement flags, and ids of external systems a test needs; never sizes, regions, or other config as env vars
Lifecycle by defaultrg -c 'skipIf' test/<P> against the file countmost lifecycles ungated
Accept-anything assertionsrg -n 'expect\(\[' test/<P>each entitlement probe asserts one specific tag
Always-true assertionsrg -n 'Array\.isArray\(.*\?\? \[\]\)|toBeDefined\(\)' test/<P>each one asserts a real value
Early exitsrg -n '^\s+return; test/<P>0
Bindings as the hostevery Bindings.test.tsdeploys a fixture host and calls the binding over HTTP; at least one suite runs the binding on a foreign host (e.g. a Cloudflare Worker)
Redeploy coveragetests for anything that syncs grants or bindings on the hostdeploy twice and assert the grant survives
Leaksafter a full run, list owned resources in the test accountnone
Slow test casesthe runner's ✓/✗ result lines, which end with each test's duration (pnpm test:integration test/<P> ... | rg '[✓✗]')no failure-path case runs longer than a few minutes
Blocked rowsrg -n 'blocked' processes/<P>/catalog/INDEX.mdeach needs payment, vendor approval, or hardware and names the exact request; quotas and preview features were requested through the API
Suite shapehow the suite was runone process, whole provider directory; quotas shared with semaphores, never batched runs

8. Docs, naming, and scope

CheckCommandExpect
Undeclared names in examplesrg -n 'existing\.' src/<P> and read every **Example:**every identifier is declared in the example
Binding examplescount @binding blocks vs those showing the host and Effect.provide(...)all of them
Resource examplesevery @resource has at least one exampleyes
Runtime JSDocrg -c '\*\*Example:\*\*' <runtime file> and its ### headings next to packages/aws/src/Lambda/Function.tsten or more examples; intro, cautions, each form, configuration, packaging, a bindings section per service, event sources
Fat-head JSDocthe same for each fat-head resource next to packages/aws/src/S3/Bucket.ts / packages/cloudflare/src/R2/Bucket.tsfive or more sections: creating, each configurable feature, binding to a runtime, runtime operations, event sources where they apply
Long-tail JSDocsample 20 long-tail filesintro with the cloud's type name, one complete creation example, a caution per replacing prop
Prop JSDocrg -n '^ [a-zA-Z]+\??:' src/<P>/<Service>/<Resource>.ts minus documented onesevery prop and attribute documented, with @default where there is one
Namespace casingls src/<P>PascalCase
Singular namesread the list of exported resource namescorrect English (Index, Process, Batch)
Scoperead the service listno end-user data, OAuth-scoped, retired, or duplicate APIs; out-of-scope list on the overview page
PR descriptionread ita ### section with a snippet for every runtime, event source, websites, and each fat-head resource; long tail in one line; no snippet claims behavior no test proves
Docs buildpnpm docs:check-jsdoc && pnpm docs:gen && pnpm docs:checkclean
Hub structurels website/src/content/docs/<p>/*/ next to ls website/src/content/docs/aws/*/Overview, Setup, tutorial/part-1..5, compute/choosing-a-runtime, sections by concern; no guides/; every AWS section has a counterpart unless the provider has nothing for it
Hub snippetsread every fenced block with import linesa complete file with a // path first line; no ..., no module-scope yield*; code matches an examples/<p>-* project or a JSDoc example
Hub voicerg -n 'Alchemy (mints|synthesizes|reconciles)|this page|three ideas|under the hood' website/src/content/docs/<p>0; second person, one or two lines per snippet
Hub wiringsidebar in website/astro.config.ts, GROUP_ICONS, docs-tabs.ts, ProviderDirectory.astro, generate-llms-txt.tsevery section listed in each; moved pages redirected in website/src/worker.ts
test/\u003cP>` | 0 |\n| Bindings as the host | every `Bindings.test.ts` | deploys a fixture host and calls the binding over HTTP; at least one suite runs the binding on a foreign host (e.g. a Cloudflare Worker) |\n| Redeploy coverage | tests for anything that syncs grants or bindings on the host | deploy twice and assert the grant survives |\n| Leaks | after a full run, list owned resources in the test account | none |\n| Slow test cases | the runner's `✓`/`✗` result lines, which end with each test's duration (`pnpm test:integration test/\u003cP> ... \\| rg '[✓✗]'`) | no failure-path case runs longer than a few minutes |\n| Blocked rows | `rg -n 'blocked' processes/\u003cP>/catalog/INDEX.md` | each needs payment, vendor approval, or hardware and names the exact request; quotas and preview features were requested through the API |\n| Suite shape | how the suite was run | one process, whole provider directory; quotas shared with semaphores, never batched runs |\n\n## 8. Docs, naming, and scope\n\n| Check | Command | Expect |\n| --- | --- | --- |\n| Undeclared names in examples | `rg -n 'existing\\.' src/\u003cP>` and read every `**Example:**` | every identifier is declared in the example |\n| Binding examples | count `@binding` blocks vs those showing the host and `Effect.provide(...)` | all of them |\n| Resource examples | every `@resource` has at least one example | yes |\n| Runtime JSDoc | `rg -c '\\*\\*Example:\\*\\*' \u003cruntime file>` and its `###` headings next to `packages/aws/src/Lambda/Function.ts` | ten or more examples; intro, cautions, each form, configuration, packaging, a bindings section per service, event sources |\n| Fat-head JSDoc | the same for each fat-head resource next to `packages/aws/src/S3/Bucket.ts` / `packages/cloudflare/src/R2/Bucket.ts` | five or more sections: creating, each configurable feature, binding to a runtime, runtime operations, event sources where they apply |\n| Long-tail JSDoc | sample 20 long-tail files | intro with the cloud's type name, one complete creation example, a caution per replacing prop |\n| Prop JSDoc | `rg -n '^ [a-zA-Z]+\\??:' src/\u003cP>/\u003cService>/\u003cResource>.ts` minus documented ones | every prop and attribute documented, with `@default` where there is one |\n| Namespace casing | `ls src/\u003cP>` | PascalCase |\n| Singular names | read the list of exported resource names | correct English (`Index`, `Process`, `Batch`) |\n| Scope | read the service list | no end-user data, OAuth-scoped, retired, or duplicate APIs; out-of-scope list on the overview page |\n| PR description | read it | a `###` section with a snippet for every runtime, event source, websites, and each fat-head resource; long tail in one line; no snippet claims behavior no test proves |\n| Docs build | `pnpm docs:check-jsdoc && pnpm docs:gen && pnpm docs:check` | clean |\n| Hub structure | `ls website/src/content/docs/\u003cp>/*/` next to `ls website/src/content/docs/aws/*/` | Overview, Setup, `tutorial/part-1..5`, `compute/choosing-a-runtime`, sections by concern; no `guides/`; every AWS section has a counterpart unless the provider has nothing for it |\n| Hub snippets | read every fenced block with `import` lines | a complete file with a `// path` first line; no `...`, no module-scope `yield*`; code matches an `examples/\u003cp>-*` project or a JSDoc example |\n| Hub voice | `rg -n 'Alchemy (mints\\|synthesizes\\|reconciles)\\|this page\\|three ideas\\|under the hood' website/src/content/docs/\u003cp>` | 0; second person, one or two lines per snippet |\n| Hub wiring | sidebar in `website/astro.config.ts`, `GROUP_ICONS`, `docs-tabs.ts`, `ProviderDirectory.astro`, `generate-llms-txt.ts` | every section listed in each; moved pages redirected in `website/src/worker.ts` |\n","preview":null,"history":null,"version":"afa061b15eeb71ec3ed0cd68606c9eb25597d012"},"/_session?shell=1":null}