Skip to content

P3 host plugin loader#5348

Open
ricochet wants to merge 3 commits into
mainfrom
p3-host-plugin-loader
Open

P3 host plugin loader#5348
ricochet wants to merge 3 commits into
mainfrom
p3-host-plugin-loader

Conversation

@ricochet

@ricochet ricochet commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

#5335 added the ComponentHostPlugin adapter, a WebAssembly component that provides a host capability from its own supervised store but nothing could actually load one outside of tests. This PR adds the production loading path across all three host front-ends, plus an end-to-end test.

A host component plugin is loaded exactly like a built-in Rust plugin (HostBuilder::with_plugin, started at Host::start); the only new work is turning a declaration (an id + a place to fetch the wasm) into a registered plugin.

Runtime primitive (wash-runtime)

  • ComponentPluginSpec / PluginSource — a plain, always-compiled declaration (id + OCI-image-or-file source + optional max_restarts / digest pin), with a FromStr for the CLI grammar.
  • load_component_plugin (gated on host-component-plugins) — fetches the wasm (oci::pull_component or tokio::fs::read), verifies an optional digest pin, and builds the ComponentHostPlugin. The caller registers it with with_plugin before Host::start.

wash host (cluster / operator path)

  • --host-plugin / WASH_HOST_PLUGINS (repeatable). Comma-separated key=value: id=<name>,image=<ref>[,pull=…][,max-restarts=N][,digest=…] or id=<name>,file=<path>.

wash dev (local path)

  • dev.host_plugins config entries (file or image source).

Chart (operator path)

  • Per-host-group runtime.hostGroups[].hostPlugins values, rendered into --host-plugin args on the host Deployment.
# wash dev — .wash/config.yaml
dev:
  host_plugins:
    - id: acme-kv
      file: ./build/kv_plugin.wasm
      maxRestarts: 3

# chart — values.yaml
runtime:
  hostGroups:
    - name: default
      hostPlugins:
        - id: acme-kv
          image: ghcr.io/acme/kv-host:1.0.0
          pullPolicy: ifNotPresent

Design decisions

  • Host-operator controlled only. A host component plugin runs with the host's capability surface and serves every workload that imports its interface — a privileged install. So it's declared in host config; nothing in a WorkloadStartRequest can register a host-global provider.
  • Static at host start. There is no dynamic-plugin API (HostApi is only heartbeat/workload_start/status/stop) and the registry is fixed after build(). Loading resolves once at boot. Dynamic load/unload on a running host is noted as future work (needs a mutable registry, workload re-bind, and an Arc<str> id).
  • All-features-only, and it fails loud. The loader is gated on host-component-plugins (not in release builds). The spec/flag/config types are always compiled, so a plugin declared against a release host produces a clear "requires the feature" error rather than a silent drop — important for the operator/env path.
  • Host CRD is status-only. It's created from the host's heartbeat (HostPodReconciler only bridges pod↔Host lifecycle); host pods are Helm-templated per host-group. So the declarative host-config seam is the host-group values, which is where hostPlugins lives.
  • &'static str id is interned via a bounded Box::leak (plugins load once at startup and live for the process); flagged to revisit if dynamic loading lands.

Testing

  • Unit: ComponentPluginSpec grammar parsing (valid/invalid).
  • Integration: load_component_plugin from a file builds a plugin with the right id/world; digest-pin-on-file and missing-file error paths; and an opt-in OCI-source test (HOST_PLUGIN_OCI_REF, self-skips in CI).
  • Config: HostPluginConfig::to_spec validation + dev.host_plugins YAML round-trip.
  • e2e (runtime-operator/test/e2e/host_plugin_test.go): a ginkgo/kind spec that loads a plugin onto a hostGroup via hostPlugins, deploys a workload importing the plugin's acme:kv/store, and asserts a value written via /set reads back via /get — proving the CLI/chart loading path and cross-store bind work in a real cluster.

Verified end-to-end locally in kind against a registry:2 on the kind network: built the all-features host image, pushed the two fixtures with wash oci push, and ran the ginkgo spec — the host pulled the plugin from OCI, loaded it, an acme:kv workload scheduled and reached Ready, and /set/get round-tripped a value through the plugin's store. SUCCESS! -- 1 Passed. The gated OCI-source load_component_plugin test and a direct wash host --host-plugin … run were also verified against the same registry. In CI this stays gated on the two fixtures being published (HOST_PLUGIN_E2E_IMAGE, HOST_PLUGIN_CALLER_E2E_IMAGE).

Follow-up to activate the e2e

The e2e self-skips until two fixtures are published (same bootstrap the implements/messaging specs used), because host plugins load at host startup. A non-pullable image would fail host boot and break the all-features leg. To turn it on:

  1. Publish crates/wash-runtime/tests/fixtures/kv-plugin and kv-plugin-caller as OCI images.
  2. Fill the empty host_plugin_image / host_plugin_caller_image matrix values in wash.yml (all-features leg).

@ricochet
ricochet requested review from a team as code owners July 15, 2026 16:39
@ricochet
ricochet force-pushed the p3-host-plugin-loader branch 2 times, most recently from 61de1c8 to 698d2ed Compare July 16, 2026 14:03
@ricochet
ricochet requested a review from a team as a code owner July 16, 2026 14:03
@ricochet
ricochet force-pushed the p3-host-plugin-loader branch 4 times, most recently from 92026d5 to f38ae8d Compare July 21, 2026 20:32
{{- /* Host component plugins: one --host-plugin flag per entry, each
rendered into the CLI's `key=value` grammar. */}}
{{- range .hostPlugins }}
- "--host-plugin=id={{ .id }}{{ if .image }},image={{ .image }}{{ if .pullPolicy }},pull={{ .pullPolicy }}{{ end }}{{ if .digest }},digest={{ .digest }}{{ end }}{{ else if .file }},file={{ .file }}{{ end }}{{ if .maxRestarts }},max-restarts={{ .maxRestarts }}{{ end }}"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI I opted to have this be plain image refs and not Artifacts.

Host is a status-only CRD (heartbeat-derived, OSArch, Version, LastSeen…), and there's no HostGroup CRD. Today these are just helm-templated into the host Deployment.

So if we want this in the future we'd essentially create a HostGroup CRD (or a HostSpec) whose hostPlugins[] carry artifactFrom references instead of inline images. Then the host reconciler:

  1. watches HostGroup + the referenced plugin Artifacts
  2. for each plugin, waits for its Artifact to reach Published
  3. renders the resolved Status.ArtifactURL into the host Deployment's --host-plugin args
  4. owns the host Deployment, so there's a single owner and re-rendering on a new artifact rolls the pod cleanly.

AKA this is something we would add on top if we wanted it.

@ricochet
ricochet force-pushed the p3-host-plugin-loader branch from f38ae8d to c80a3ef Compare July 22, 2026 00:35
vados-cosmonic
vados-cosmonic previously approved these changes Jul 22, 2026

@vados-cosmonic vados-cosmonic left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

Nothing but nits!

Comment thread charts/runtime-operator/values.yaml
Comment thread crates/wash-runtime/src/plugin/component_plugin_spec.rs
Comment thread crates/wash-runtime/src/plugin/component_plugin_spec.rs Outdated
Comment thread crates/wash-runtime/src/plugin/component_plugin_spec.rs Outdated
Comment thread crates/wash/src/cli/host.rs Outdated
Comment thread crates/wash/src/cli/host.rs Outdated
ricochet added 3 commits July 22, 2026 18:04
Wire the existing `ComponentHostPlugin` adapter to `wash host`, `wash dev`,
and the runtime-operator chart so host component plugins — WebAssembly
components that provide a host capability — can be loaded declaratively.

- wash-runtime: `ComponentPluginSpec`/`PluginSource` (always compiled) plus a
  feature-gated `load_component_plugin` that fetches the wasm (OCI pull or
  local file), verifies an optional digest pin, and builds the plugin.
- wash host: `--host-plugin`/`WASH_HOST_PLUGINS` (repeatable, `key=value`
  grammar). Gated wiring that fails clearly on a build without the
  `host-component-plugins` feature rather than silently dropping the request.
  Private-registry plugin pulls authenticate with `--host-plugin-registry-user`/
  `--host-plugin-registry-password` (env `WASH_HOST_PLUGIN_REGISTRY_USER`/
  `WASH_HOST_PLUGIN_REGISTRY_PASSWORD`, required together, `hide_env_values` +
  sourced from the environment so the secret stays out of the `--host-plugin`
  arg and the pod spec), falling back to the ambient docker credential helper
  and then anonymous access. Workload components keep their own per-workload
  image pull secret.
- wash dev: `dev.host_plugins` config entries (file or image source).
- chart: per-host-group `hostPlugins` values rendered into `--host-plugin` args.

Loading is host-operator controlled only; nothing in a workload request can
register a host-global capability provider. The loader stays gated on
`host-component-plugins` (all-features builds only) — a release host handed a
plugin it cannot build fails to start. The gated wash unit tests
(`cargo test -p wash --lib --features host-component-plugins`) run in CI
alongside the wash-runtime feature tests.

Signed-off-by: Bailey Hayes <bailey@cosmonic.com>
The gateway's workloadHostname registered a Host-header route only for
workloads declaring the p2 `wasi:http/incoming-handler` interface. A workload
that exports the p3 `wasi:http/handler@0.3.0` — which the host serves all the
same (wash-runtime's is_incoming_http_handler accepts either) — reached Ready
but got no gateway route, so every request to it 503'd at the gateway with no
trace on the host.

Register a route for either interface, matching what the host serves. Covered
by a workloadHostname unit test (p2, p3, missing host config, non-http).

Signed-off-by: Bailey Hayes <bailey@cosmonic.com>
Add an operator e2e spec that proves the host-plugin loading path end to end in
a real cluster: a hostGroup `hostPlugins` entry renders to `--host-plugin`,
`wash host` fetches and registers the plugin at startup, a workload importing
the plugin's bespoke `acme:kv/store` interface binds across the store boundary,
and a value written via `/set` reads back via `/get` from the plugin's store.

Built on the in-cluster registry flow (make e2e-images), like the implements and
messaging specs, rather than out-of-band published images:

- xtask e2e-images: build and push the kv-plugin and kv-plugin-caller fixtures
  into the in-cluster oci-registry. FIXTURES now carries each fixture's
  FixtureKind so the P3 (async) plugin fixtures resolve to their wasm32-wasip1
  build output rather than the wasip2 path the plain components use.
- host_plugin_test.go: pull both fixtures by their in-cluster registryRef; gate
  on the in-cluster registry AND an all-features fixture host (the
  host-component-plugins feature is off in release builds), self-skipping
  otherwise. A plugin loads at host startup, so BeforeAll rolls the default
  hostgroup with the plugin via `helm upgrade` (the registry is serving by then)
  and AfterAll restores a plugin-free host, keeping the spec order-independent.
  The caller exposes its capability over the P3 wasi:http/handler@0.3.0 export.
- e2e_suite_test.go: the local BUILD_RUNTIME_IMAGE build enables
  host-component-plugins alongside implements, matching the all-features
  wash-image build; it also builds the runtime-gateway from the local tree (and
  points gateway.image at it) so branch changes to the gateway — like routing
  the p3 handler above — are exercised instead of the published canary.

Signed-off-by: Bailey Hayes <bailey@cosmonic.com>
@ricochet
ricochet force-pushed the p3-host-plugin-loader branch from 8202688 to 5cd52cc Compare July 22, 2026 22:04
@ricochet
ricochet enabled auto-merge (rebase) July 22, 2026 22:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants