Skip to content

Restore pre-merge Venafi coverage using a vendor-supplied verified fake #1226

Description

@wallrj

#1225 removes the optional /test-only per-issuer Venafi presubmits, because presubmits run unreviewed PR code and must not carry live credentials. That leaves no pre-merge way to exercise the Venafi issuers: the only remaining coverage is the ci-*-issuers-venafi periodic (every 2h on master, every 12h on release branches), so a Venafi regression now only surfaces after merge, and a contributor fixing a Venafi bug has no CI job with which to demonstrate the fix.

Proposal

A verified fake Venafi API server, supplied by Venafi / CyberArk / Palo Alto Networks for use by any software integrating with their APIs, not just cert-manager:

  • Accurately simulates the TPP, Cloud and NGTS APIs, and can be controlled and seeded via a sidechannel API.
  • Verified by a conformance suite which runs periodically against both the real and the fake APIs, demonstrating the fake remains accurate as the real APIs evolve. The runs against real servers need secrets, but they are periodics, completely isolated from PR jobs (see Add a dedicated node pool for credentialed Prow jobs infrastructure#89 for node-level isolation of credentialed jobs).
  • cert-manager then re-adds per-issuer Venafi presubmits pointed at the fake — pre-merge coverage restored, credential-free.

Prior art

  • letsencrypt/pebble: the provider-maintained fake ACME server which cert-manager's e2e presubmits already use for credential-free ACME coverage. The model works because the API owner maintains the fake.
  • Pact provider verification: the verification loop — a contract suite run cheaply against the fake pre-merge, and periodically against the real provider to prove the contract still holds.

Caveats

  • The conformance suite is the spec: it must cover the behaviours integrators actually depend on (auth flows, error shapes, async issuance), not just happy paths.
  • The conformance periodic needs an owner and an alert: a stale fake with green presubmits is misplaced confidence, worse than known-absent coverage.

with claude fable-5

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions