Skip to content

Reinstate storage driver domain - #7169

Open
MarkSymsCtx wants to merge 2 commits into
xapi-project:masterfrom
MarkSymsCtx:reinstate_storage_driver_domain
Open

Reinstate storage driver domain#7169
MarkSymsCtx wants to merge 2 commits into
xapi-project:masterfrom
MarkSymsCtx:reinstate_storage_driver_domain

Conversation

@MarkSymsCtx

@MarkSymsCtx MarkSymsCtx commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

No description provided.

@MarkSymsCtx

MarkSymsCtx commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

This has completed XenRT BST 246161 with just a transient in-flight issue around new guest templates as well as specific Storage Driver Domain testing.

psafont
psafont reviewed Jul 9, 2026
View reviewed changes
Comment thread ocaml/tests/test_pbd.ml Outdated
@MarkSymsCtx
MarkSymsCtx requested a review from psafont July 9, 2026 12:12
@last-genius

last-genius commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

What's the motivation and context behind this? Does something actually use non-dom0 storage domains? Are there plans for them? I'm wary of adding something that's going to sit unused and untested

@MarkSymsCtx

MarkSymsCtx commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

What's the motivation and context behind this? Does something actually use non-dom0 storage domains? Are there plans for them? I'm wary of adding something that's going to sit unused and untested

The gap was identified while performing feasibility study work for possible future storage implementations. We have (and have had for a good while) various bits of prototype code that use storage driver domain semantics as well as tests that can exercise it in XenRT. We have had more than one storage partner interested at various times about implementing something that would require or benefit from this functionality and sometimes it can just be a useful thing to takes something that is generically available as a Linux appliance and allow it to be used as a storage provider without needing to do a full rebuild of componentry or drivers into XenServer/XCP-ng.

last-genius
last-genius requested changes Jul 9, 2026
View reviewed changes
Comment thread ocaml/idl/datamodel.ml Outdated
Comment thread ocaml/idl/datamodel.ml Outdated
Comment thread ocaml/tests/test_pbd.ml Outdated
Comment thread ocaml/xapi/xapi_xenops.ml
Comment thread ocaml/xapi/xapi_vm_lifecycle.ml
minglumlu
minglumlu reviewed Jul 10, 2026
View reviewed changes
Comment thread ocaml/idl/datamodel.ml
minglumlu
minglumlu reviewed Jul 10, 2026
View reviewed changes
Comment thread ocaml/xapi/xapi_pbd.ml
minglumlu
minglumlu reviewed Jul 10, 2026
View reviewed changes
Comment thread ocaml/xapi/xapi_pbd.ml
minglumlu
minglumlu reviewed Jul 10, 2026
View reviewed changes
Comment thread ocaml/xapi/xapi_vm_helpers.ml
minglumlu
minglumlu reviewed Jul 15, 2026
View reviewed changes
Comment thread ocaml/xapi/system_domains.ml Outdated
@MarkSymsCtx
MarkSymsCtx force-pushed the reinstate_storage_driver_domain branch from 2a676b3 to d546b17 Compare July 20, 2026 16:12
@MarkSymsCtx @claude
xapi: Add PBD.storage_driver_domain and honour it at domain create
3d46370 
Reinstate the storage driver domain capability removed in xapi-project#7033
(XSA-489 / CVE-2026-23561), which dropped recognition of the
PBD.other_config:storage_driver_domain key, as a first-class,
RBAC-protected field with the supporting runtime behaviour.

- IDL: add a DynamicRO PBD.storage_driver_domain field (Ref VM; null means
  dom0) and a Pool-Admin-only PBD.set_storage_driver_domain message whose
  implementation rejects a value that does not reference an existing object
  (INVALID_VALUE) rather than silently storing an unresolvable ref.

- system_domains.ml: resolve storage_driver_domain_of_pbd / _of_vbd /
  pbd_of_vm from the new field so storage_access.bind starts the driver
  domain and routes the SR backend RPC to it, as before the key was dropped.

- xe CLI: expose storage-driver-domain in pbd_record (settable by VM uuid,
  null renders as OpaqueRef:NULL).

- xapi_xenops.ml: a VM acting as a storage driver domain (i.e. referenced by
  some PBD's storage_driver_domain field) is created with ssidref 123000 and,
  unless platformdata already specifies one, max_maptrack_frames = 1024 so
  Xen permits it to map grant references from the VMs it is serving storage
  to. Other VMs are unchanged.

- Add unit tests (ocaml/tests/test_pbd.ml): default-null, set/clear via
  Xapi_pbd.set_storage_driver_domain, and invalid-ref rejection.

Bumps the datamodel schema version and hash accordingly.

Signed-off-by: Mark Syms <mark.syms@citrix.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@MarkSymsCtx
MarkSymsCtx force-pushed the reinstate_storage_driver_domain branch from d546b17 to e3f465a Compare July 21, 2026 08:25
last-genius
last-genius approved these changes Jul 22, 2026
View reviewed changes
Comment thread ocaml/tests/test_pbd.ml
@MarkSymsCtx @claude
xapi: Constrain lifecycle and placement of storage driver domains
32c8ee9 
A VM acting as a storage driver domain (referenced by one or more PBDs via
PBD.storage_driver_domain) backs storage for the guests those PBDs serve.
Disrupting or moving it breaks I/O for every VM it serves, so:

- While any associated PBD is plugged, refuse clean/hard shutdown,
  clean/hard reboot, suspend, pause, and checkpoint.
- Pin it to the host owning its PBD(s): it may only boot there.
- Never allow it to migrate (pool_migrate / migrate_send), even when halted.

The lifecycle guards live in Xapi_vm_lifecycle.check_operation_error, so they
cover both VM.allowed_operations and the runtime assert_operation_valid used
by every lifecycle call. The host pin is enforced in
Xapi_vm_helpers.assert_can_boot_here (modelled on the existing control-domain
affinity pin), and migration is additionally refused in
Xapi_vm_migrate.assert_can_migrate. All blocks report OPERATION_NOT_ALLOWED
with a descriptive message.

Adds System_domains.pbds_of_vm / is_storage_driver_domain, and unit tests in
ocaml/tests/test_pbd.ml.

Signed-off-by: Mark Syms <mark.syms@citrix.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@MarkSymsCtx
MarkSymsCtx force-pushed the reinstate_storage_driver_domain branch from e3f465a to 32c8ee9 Compare July 22, 2026 13:46
@TSnake41

TSnake41 commented Jul 23, 2026 •
edited
Loading

Copy link
Copy Markdown

How would that interact with the needs of qemu-dm ?
While I think it would be fine with PVH domains as they only require a backend towards the guest (thus only guest<->storage domain); for the "HVM" case, qemu running in the Dom0 also needs to access the disk directly (which we currently do with nbd), but it gets a bit more complicated if it's actually coming from another VM.

@MarkSymsCtx

MarkSymsCtx commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

How would that interact with the needs of qemu-dm ? While I think it would be fine with PVH domains as they only require a backend towards the guest (thus only guest<->storage domain); for the "HVM" case, qemu running in the Dom0 also needs to access the disk directly (which we currently do with nbd), but it gets a bit more complicated if it's actually coming from another VM.

It's obviously implementation specific but there are several options

  • if the driver domain VM is using xen-blkback as the I/O datapath the PV device is also presented to Dom0 and thus to qemu-dm. This is what the simple test that we have used to validate this code with does and it is possible to both attach a VDI presented by the driver domain as a hotplug and also to perform an install of an HVM guest. This probably holds for using any other component that interfaces with the Xen PV rings and event channels but this has not be validated.
  • If the driver domain VM can provide an NBD server then it can respond with an NBD URI. You still need to have some code in Dom0 to interface Xapi to the Storage Controller but this can in most cases be a thin shim that forwards requests to the driver domain and then propagates, possibly rewriting (e.g. adjusting URI targets to include the details of the domain), results back.

Ultimately it is the responsibility of the driver domain implementation to ensure that it satisfies the requirements of the toolstack upon the Storage API. Probably something based on SMAPIv3 as that ensures a cleaner split between toolstack and storage layers and does not permit layering violations like those that exist in all SMAPIv1 implementations. It probably would be possible to implement something to the SMAPIv1 XML RPC but I certainly haven't investigated that when it is relatively straightforward to implement something under SMAPIv3.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@minglumlu minglumlu minglumlu approved these changes
@last-genius last-genius last-genius approved these changes
@cplaursen cplaursen Awaiting requested review from cplaursen
@liulinC liulinC Awaiting requested review from liulinC
@LunfanZhang LunfanZhang Awaiting requested review from LunfanZhang
@changlei-li changlei-li Awaiting requested review from changlei-li
@psafont psafont Awaiting requested review from psafont

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@MarkSymsCtx @last-genius @TSnake41 @psafont @minglumlu