Bug Report: injectOwnerReferenceHandler rejects protobuf request bodies, breaking CLI writes through the proxy

[tiyajp]

Summary

The ansible-operator proxy's injectOwnerReferenceHandler assumes all write request bodies are JSON. When a client sends a create/update with Content-Type: application/vnd.kubernetes.protobuf, the handler fails to deserialize the body and rejects the request. As a result, any resource created through the proxy by a client that defaults to protobuf — notably recent oc builds (4.20+) invoked from within Ansible tasks — fails outright.

Because newer OpenShift CLI releases changed their default request serialization from JSON to protobuf, this silently breaks previously-working operators when only the CLI version in the operator image changes.

Environment

• ansible-operator-plugins (bundled in ansible-operator base image)
• Client inside the operator container: oc 4.20 (sends Content-Type: application/vnd.kubernetes.protobuf)
• Same tasks work with oc 4.18 (sends Content-Type: application/json)

Steps to Reproduce

1. Build an Ansible-based operator whose playbook shells out to the CLI for a write, e.g.:

  - name: Create a secret via CLI
    ansible.builtin.shell: "oc create secret generic test-secret -n {{ namespace }} --from-literal=a=b"   

2. Use an operator image containing oc >= 4.20 (protobuf-by-default client).
3. Trigger a reconcile.

Expected Behavior

Either:

• the proxy deserializes the protobuf body, injects the ownerReference, and forwards the request; or
• at minimum, the proxy performs content negotiation and handles non-JSON bodies deliberately (documented pass-through with a warning), rather than failing the request.

Actual Behavior

The handler attempts json deserialization on the protobuf body. The Kubernetes protobuf wire format begins with the 4-byte magic prefix k8s\x00 (0x6b 0x38 0x73 0x00), so JSON parsing fails on the first byte:

{"level":"info","logger":"proxy","msg":"Injecting owner reference"}
{"level":"error","logger":"proxy","msg":"Could not deserialize request body",
 "error":"invalid character 'k' looking for beginning of value",
 "stacktrace":"...internal/ansible/proxy.(*injectOwnerReferenceHandler).ServeHTTP
    /host/internal/ansible/proxy/inject_owner.go:123
  ...internal/ansible/proxy.(*cacheResponseHandler).ServeHTTP
    /host/internal/ansible/proxy/cache_response.go:150"}

The create is rejected at the proxy; the resource is never sent to the API server. Reads (GET) are unaffected since they carry no request body, which makes the failure hard to diagnose: existence checks succeed while creates fail, and downstream workloads fail much later with missing-resource symptoms.

Impact

• Any Ansible operator that invokes oc/kubectl for writes breaks when the bundled CLI moves to a protobuf-by-default version. Pinning the older CLI is not viable long-term (newer releases carry CVE fixes).
• The failure mode is misleading: the error surfaces only in operator pod logs under the proxy logger, while the Ansible task may appear to succeed depending on script error handling, and the visible failure occurs in unrelated workloads that consume the missing resources.

Root Cause

injectOwnerReferenceHandler.ServeHTTP (internal/ansible/proxy/inject_owner.go) unconditionally treats the request body as JSON. It performs no Content-Type inspection and has no protobuf (application/vnd.kubernetes.protobuf) decoding path, although the Kubernetes API — and therefore clients talking through the proxy — fully supports protobuf request bodies.

Suggested Fix Options

1. Content negotiation in the injection handler: inspect Content-Type; decode protobuf bodies using the apimachinery protobuf serializer, inject the ownerReference, re-encode, and forward. This preserves owner injection for all clients.
2. Documented pass-through: if decoding is unsupported for a given content type, forward the request unmodified and log a clear warning that ownerReferences were not injected (trade-off: silently orphaned resources — likely worth gating behind an option).
3. At minimum, an explicit error message: detect the k8s\x00 protobuf magic prefix and return an actionable error ("proxy cannot inject owner references into protobuf-encoded requests; use JSON-encoding clients such as the k8s modules"), instead of a generic JSON parse error.

Option 1 seems most consistent with the proxy's purpose. Option 3 is cheap and would have cut diagnosis time significantly.

[tiyajp]

We're migrating our roles to kubernetes.core.k8s modules as a workaround, but wanted to ask: is a fix for the proxy planned (content negotiation or protobuf decoding in the injection handler), and if so, is there a target release? This will increasingly affect anyone bundling recent oc/kubectl builds, so it would help us plan whether to treat the module migration as temporary mitigation or the permanent state.