Malformed nfsPath on TridentVolume CR under concurrent ontap-nas-economy provisioning (missing qtree pool prefix)

[nikmitra]

Summary

When provisioning 100 PVCs concurrently via Trident CSI using the ontap-nas-economy storage driver, all volumes were created successfully on ONTAP and reported ProvisioningSucceeded, but 9 of 100 TridentVolume CRs were persisted with a malformed nfsPath missing the flexvol/qtree-pool prefix. Pods mounting those volumes failed with NFS mount exit status 32.

The qtree and Internal ID on ONTAP appear correct; the issue is in the NFS export path stored on the TridentVolume CR and passed to the node driver at mount time. This looks like a race or logic bug during NFS path construction under high concurrency (ConstructOntapNASQTreeVolumePath), not a backend provisioning failure.

---

Environment

Item	Value
Trident version	26.06.0
Trident store	crdv1
Storage driver	ontap-nas-economy
Access mode	ReadWriteMany (RWX)
Qtrees per flexvol	100
Limit aggregate usage	70% (configured on backend)
Platform	Kubernetes / OpenShift

---

Steps to reproduce

1. Configure a Trident backend with storageDriverName: ontap-nas-economy and qtree pooling enabled.
2. Create 100 PVCs concurrently using the same storage class (e.g. 1Gi each, RWX).
3. Wait for all PVCs to bind and TridentVolume CRs to reach online.
4. Schedule pods that mount each PVC.
5. Observe mount failures on a subset of volumes.

---

Expected behavior

Every TridentVolume CR should have a complete NFS export path including the flexvol/qtree-pool prefix, for example:

/<qtree_pool_flexvol_name>/<qtree_internal_name>

Node mounts should succeed using:

<nfs_server>:/<qtree_pool_flexvol_name>/<qtree_internal_name>

---

Actual behavior

• 100/100 PVCs: CSI ProvisioningSucceeded, TridentVolume CR State: online
• 91/100 volumes: correct nfsPath, mounts succeed
• 9/100 volumes: malformed nfsPath stored as //<qtree_internal_name> (missing flexvol pool prefix)
• Affected pods fail at NodePublishVolume with:

error mounting NFS volume <nfs_server>://<qtree_internal_name>
on mountpoint .../pvc-<uuid>/mount: exit status 32

---

Affected volumes (example pattern)

All 9 failures followed the same pattern: valid Internal ID, invalid nfsPath.

Status	Field	Example
Correct	Internal ID	/svm/<svm>/flexvol/<qtree_pool_flexvol>/qtree/<qtree_internal_name>
Incorrect	nfsPath	//<qtree_internal_name>
Expected	nfsPath	/<qtree_pool_flexvol>/<qtree_internal_name>

Example TridentVolume CR (bad):

Config:
  Access Information:
    Nfs Path: //<qtree_internal_name>                    # WRONG — missing flexvol prefix
    Nfs Server Ip: <nfs_server>
  Internal ID: /svm/<svm>/flexvol/<qtree_pool_flexvol>/qtree/<qtree_internal_name>  # CORRECT
State: online

Example TridentVolume CR (good):

Config:
  Access Information:
    Nfs Path: /<qtree_pool_flexvol>/<qtree_internal_name>  # CORRECT
State: online

---

Evidence

Trident controller

During CreateVolume, qtree creation completes successfully, then path construction runs:

Qtree created after 0.03 seconds. Name=<qtree_internal_name>
>>>> smb.ConstructOntapNASQTreeVolumePath
<<<< smb.ConstructOntapNASQTreeVolumePath
AddVolume ... volume.Name=pvc-<uuid>

Despite ConstructOntapNASQTreeVolumePath being invoked, the persisted TridentVolume CR ends up with //<qtree_internal_name> instead of /<qtree_pool_flexvol>/<qtree_internal_name>.

Trident node (CSI)

level=error msg="GRPC error: rpc error: code = Internal desc = error mounting NFS volume
<nfs_server>://<qtree_internal_name>
on mountpoint .../pvc-<uuid>/mount: exit status 32"

Exit status 32 indicates NFS mount failure (invalid or non-existent export path).

---

Root cause hypothesis

Under concurrent qtree creation on ontap-nas-economy, ConstructOntapNASQTreeVolumePath (or the code path that sets Config.AccessInformation.NfsPath on the TridentVolume CR) sometimes returns or persists a path without the flexvol/qtree-pool component, producing //<internalName> instead of /<flexvolPool>/<internalName>.

Possible contributing factors:

• Race when many CreateVolume requests run in parallel during an initial burst
• Flexvol NAS path not yet available when path construction runs
• No validation before persisting the TridentVolume CR (bad path accepted despite valid Internal ID)

This issue is distinct from aggregate capacity limits. Aggregate threshold enforcement may cause separate provisioning failures on other workloads, but the 100-PVC concurrent test described here did not fail at provisioning time — only at mount time due to the malformed path.

---

Impact

• 9% failure rate in this concurrent test (9/100)
• Volumes appear healthy (online, bound PVCs) but are unusable until nfsPath is corrected
• Failure is silent at the provisioning layer; it only surfaces when pods attempt to mount

---

Workaround

Manually patch affected TridentVolume CRs to set the correct nfsPath derived from Internal ID:

Internal ID: .../flexvol/<FLEXVOL>/qtree/<QTREE>
nfsPath should be: /<FLEXVOL>/<QTREE>

Alternatively, delete and recreate affected PVCs (the race may recur under similar concurrency).

---

Suggested fix

1. Ensure ConstructOntapNASQTreeVolumePath always includes the flexvol/qtree-pool prefix before persisting the TridentVolume CR.
2. Add validation: reject or retry if nfsPath does not match /<flexvol>/<qtree> for the economy driver.
3. Add a concurrency regression test: 100+ simultaneous PVC creates on ontap-nas-economy, verifying all nfsPath values and successful node mounts.

---

Labels (suggested)

bug, ontap-nas-economy, nfs, concurrency, csi

[sjpeeris]

Hi @nikmitra Thanks for reporting and the clear writeup. We will prioritize this in for our future release.