data-plane-controller: write private link DNS details after PulumiUp1 instead of end-of-converge #3477

What to build

A private link's endpoint is created during PulumiUp1, and est-dry-dock's link_results export carries its final AWS-assigned DNS names at that moment. The controller does not write them to internal.data_plane_private_links until the end of the Ansible stage, roughly ten minutes later (PulumiUp1, a 300s DNS propagation wait, the Ansible run, then the write drained on the next poll during PulumiUp2).

For that whole window the row reads status = 'pending' with details = null, while the endpoint already exists and its DNS name is final. Support is usually adding a link in response to a customer who is waiting on that DNS name to configure their own side, and the only way to get it early is to read the raw DPC logs for the pulumi-up-one diff. That is a poor interface for a value the control plane already has.

Write each link's details as soon as PulumiUp1 completes, so the row carries the DNS names within about a minute of the endpoint being created.

Why this looks safe

Nothing after PulumiUp1 affects a private link:

• Ansible has no private-link, VPC-endpoint, or vpce references at all. It configures servers; the endpoint is a VPC-level resource with no host-side configuration.
• PulumiUp2 is scoped to scaling: it updates DNS records for resources that have now started and destroys scaled-down resources that have now stopped. A link insert scales nothing.
• Both pulumi up invocations run the same program, so PulumiUp2 re-evaluates the endpoint and finds no diff.
• A link that can fail has already failed by PulumiUp1: PrivateLinkError is caught per link and produces a failed result with its error.
• The 300s AwaitDNS1 wait is one DNS_TTL_ACTUAL, waiting out the 300s TTL on records est-dry-dock itself publishes for replaced and scaling-down hosts. The endpoint's vpce-*.vpce.amazonaws.com name is published by AWS at creation and is not one of those records, so the wait does not gate it.

One edge case: adding the first link to a plane with no bastion and no allowed CIDRs flips uses_private_network, which changes network topology and may pull in resources Ansible then configures. Worth confirming this does not need different treatment from the steady-state case.

Sketch

The pieces this needs mostly exist. pinned_links is already pinned at exactly PulumiUp1, with a comment noting that later polls retain those pins "for attributing its exports", so the generation guard works unchanged. write_private_link_statuses already takes pinned links plus results, already guards on generation, and is idempotent, so the existing end-of-Ansible write can stay and simply overwrite with identical content.

That leaves: run pulumi stack output in on_pulumi_up_1 (the same call already exists in on_ansible), stage the parsed link_results in a new Outcome field drained the way publish_exports is, and pass it to the existing write. ControlExports already tolerates a missing link_results, so a partial stack output at this stage parses.

No schema change is needed. pending is already a legal value of the status check constraint, and details is a plain nullable jsonb column independent of it.

Open questions

These are design calls for whoever owns the private-links model, not implementation details:

1. Should status also become provisioned at PulumiUp1? The evidence above says PulumiUp1 genuinely is the moment a link is provisioned, which would make an early provisioned accurate. But the current design deliberately means "a completed converge observed this", and leaving status at pending preserves that invariant while still delivering the DNS names early. Leaving it pending is the smaller change and the safer default; confirming which is wanted should come before implementation.

2. What should observed_at do on an early write? The current UPDATE sets it to now() unconditionally alongside status, details and error. If status stays pending, stamping observed_at early contradicts "a converge observed this", so the early path likely needs to leave it null.

Acceptance criteria

• After a link's endpoint is created in PulumiUp1, its row's details carries the same dns_entries and service_name the end-of-converge write would have produced.
• The generation guard still holds: a link whose config is edited mid-converge is not stamped by the converge that pinned the earlier generation.
• The existing end-of-Ansible write still runs and is a no-op in content for links already written at PulumiUp1.
• A link that fails during PulumiUp1 still records failed with its error.
• The behaviour of status and observed_at on the early write matches whichever answer the open questions above settle on.

Blocked by

None (can start immediately), though the two open questions should be answered before implementation.