Each row turns a rule from AGENTS.md or the skill into a check with a
command and an expected result. The rows come from the first review of the
GCP provider and the corrections made during it; on the pre-fix GCP code
these greps returned hundreds of hits each.
Run it at every wave boundary and before calling a provider done. Use a
fresh reviewer (a subagent with no stake in the code) and have it report,
for every item, the command it ran, the count, and the file:line of each
hit it did not accept. "Looks fine" is not an answer. Every accepted hit
needs a one-line reason.
<P> is the provider name. Paths are relative to the repo root:
src/<P> means packages/<p>/src (the provider package) and test/<P>
means packages/integration-tests/test/<P>. The greps find candidates;
read each hit.
0. Coverage
Run this first; the other sections only judge the code that exists.
Check
Command
Expect
Every service cataloged
compare ls submodules/distilled/packages/<pkg>/src/services with ls processes/<P>/catalog
one catalog file per service
Nothing left unbuilt
count statuses in processes/<P>/catalog/INDEX.md
no missing or implemented rows; every out-of-scope and blocked row has a reason or exact error
Index matches the code
every tested row has a Resource<...>("<P>.<Service>.<Name>") in src/<P> and a registered provider in Providers.ts
yes
One test file per resource
every tested row has test/<P>/<Service>/<Name>.test.ts
yes
Operations classified
every operation in each distilled service module appears in its catalog file as resource-lifecycle, binding, event-source, or internal with a reason
yes; the provider's port of scripts/audit-service.ts prints no unclassified operations
Binding coverage
the audit's missingBindings per service, and rg -l 'Binding\.Service<' src/<P> | xargs -n1 dirname | sort -u | wc -l against the services with binding operations
no missing bindings; every service with data-plane operations has bindings (AWS: 205 of 212)
Bindings through distilled
read each *Http.ts
wraps a @distilled.cloud/<pkg> operation; no HttpClientRequest
Non-resource deliverables
index rows for runtime, event sources, websites, examples
none missing
Skill freshness
git log -1 --format=%H -- .agents/skills/alchemy-provider vs the Skill: <sha> line at the top of INDEX.md
equal; a newer skill commit means re-read it and add rows for its new deliverables
Framework examples
ls examples | grep '^<p>-website-' vs ls examples | grep '^hetzner-website-'
one per framework the provider hosts (skip when it can't host websites)
Workload examples
ls examples | grep '^<p>-' against the workload table in Step 10
one per workload the cloud supports; each registered in tsconfig.json and scripts/test-examples.ts, with a passing test/integ.test.ts
Docs generated for everything
ls website/src/content/docs/providers/<P> after pnpm docs:gen
one page per resource and binding page group, matching the index
Out-of-scope list published
the overview page's "Out of scope" section
matches the out-of-scope rows
How to compare against what good looks like
For each service, open the closest AWS or Cloudflare equivalent and compare
side by side: props naming, diff, reconcile, binding shape, test shape,
JSDoc examples. Where the new provider differs, the difference needs a
reason the platform forces. References:
every hit is a presigned/SAS URL from the API or the user's own deployed endpoint, with a comment saying so; data-plane and credential calls go through distilled
Catch-all errors handled
rg -n 'Unknown\w*Error|HttpError' src/<P>
0
Lists swallowing errors
rg -n 'orElseSucceed|catch\w*\(.*\[\]' src/<P>
0 on any list/read path
Single-page lists
read every list and findBy* helper
paginated (.items(...) or a page-token loop)
4. Lifecycle
Check
How
Expect
One operation waiter
rg -c 'waitFor\w*Operation|pollOperation' src/<P>
one definition in a shared module, many call sites
Realistic wait budgets
read each budget against the provider's documented create time
minutes for databases, clusters, environments; never a test-style 10 × 5s
Ready before return
read each reconcile
returns after the API reports ready, not after the create call
SQL migrations
for each resource hosting a SQL schema, rg -n 'runMigrations' src/<P>
a migrations prop driven by @alchemy.run/sql/Migrations (see SQL database migrations in AGENTS.md), with a live test that adds a file and redeploys
Identity change replaces
for each user-chosen name/id prop, read diff
replace; a rename test proves the old one is deleted