You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#1225 removes the optional /test-only per-issuer Venafi presubmits, because presubmits run unreviewed PR code and must not carry live credentials. That leaves no pre-merge way to exercise the Venafi issuers: the only remaining coverage is the ci-*-issuers-venafi periodic (every 2h on master, every 12h on release branches), so a Venafi regression now only surfaces after merge, and a contributor fixing a Venafi bug has no CI job with which to demonstrate the fix.
Proposal
A verified fake Venafi API server, supplied by Venafi / CyberArk / Palo Alto Networks for use by any software integrating with their APIs, not just cert-manager:
Accurately simulates the TPP, Cloud and NGTS APIs, and can be controlled and seeded via a sidechannel API.
Verified by a conformance suite which runs periodically against both the real and the fake APIs, demonstrating the fake remains accurate as the real APIs evolve. The runs against real servers need secrets, but they are periodics, completely isolated from PR jobs (see Add a dedicated node pool for credentialed Prow jobs infrastructure#89 for node-level isolation of credentialed jobs).
cert-manager then re-adds per-issuer Venafi presubmits pointed at the fake — pre-merge coverage restored, credential-free.
Prior art
letsencrypt/pebble: the provider-maintained fake ACME server which cert-manager's e2e presubmits already use for credential-free ACME coverage. The model works because the API owner maintains the fake.
Pact provider verification: the verification loop — a contract suite run cheaply against the fake pre-merge, and periodically against the real provider to prove the contract still holds.
Caveats
The conformance suite is the spec: it must cover the behaviours integrators actually depend on (auth flows, error shapes, async issuance), not just happy paths.
The conformance periodic needs an owner and an alert: a stale fake with green presubmits is misplaced confidence, worse than known-absent coverage.
#1225 removes the optional
/test-only per-issuer Venafi presubmits, because presubmits run unreviewed PR code and must not carry live credentials. That leaves no pre-merge way to exercise the Venafi issuers: the only remaining coverage is theci-*-issuers-venafiperiodic (every 2h on master, every 12h on release branches), so a Venafi regression now only surfaces after merge, and a contributor fixing a Venafi bug has no CI job with which to demonstrate the fix.Proposal
A verified fake Venafi API server, supplied by Venafi / CyberArk / Palo Alto Networks for use by any software integrating with their APIs, not just cert-manager:
Prior art
Caveats
with claude fable-5