[RFD]: Add dynamic discovery using Magellan libs, producing static-discoverable output

[synackd]

Decision Goal

Determine if dynamic discovery to automate the creation of a nodes.yaml in the CLI is necessary.

Category

Tooling

Stakeholders / Affected Areas

Users

Decision Needed By

No response

Problem Statement

The Magellan tool can dynamically discover nodes via Redfish queries, outputting verbose Redfish data to send to SMD. However, this information does not quite align with the nodes file format used with the ochami CLI's static discovery. Some sites use the nodes file to store their inventory as Infrastructure as Code, and so it would be useful in these cases to store the Magellan-discovered output in the nodes.yaml format used by the CLI, effectively dynamically generating a workable nodes.yaml for use in static discovery.

Proposed Solution

I can think of a few solutions that could address this.

SOLUTION 1: Perform dynamic discovery in the ochami CLI, outputting a nodes file

Use Magellan libraries to perform dynamic discovery using a hypothetical ochami discover dynamic command, outputting a nodes file to standard output or file path, formatting as YAML or JSON, that can be used directly with ochami discover static.

The benefit of this is that a nodes file (which can be reused) can be generated while also populating SMD, e.g:

# Dynamically discover nodes, store in nodes.yaml, and
# populate SMD with found data
ochami discover dynamic -f yaml | tee nodes.yaml | ochami discover static -F yaml

One drawback is the potential complexity added with credential management and having to pipe commands together to achieve a persistent nodes file while populating SMD. However, the current alternative is to run Magellan and manually adapt the output to a nodes file, which is quite a bit more work-intensive. This specific problem is addressed in SOLUTION 2 below.

SOLUTION 2: Convert from Magellan output to nodes file

Rather than reimplement dynamic discovery, provide a way to ingest Magellan discovery output and convert it into the nodes file format. For instance:

# Configure BMC secrets
export MASTER_KEY=$(magellan secrets generatekey)
magellan secrets add default ...

# Use Magellan for dynamic discovery
magellan scan --subnet 172.16.0.0/24

# Collect BMC Redfish info and convert to a nodes.yaml
magellan collect -F json | tee nodes-magellan.json | ochami discover convert -f yaml -o nodes.yaml

# Populate SMD with changes...
# ...using ochami CLI...
ochami discover static -F yaml -d @nodes.yaml
# ...or using Magellan
magellan send -d @nodes-magellan.json https://smd.openchami.cluster

This would eliminate the need to implement discovery logic in the ochami CLI, but may require more command incantation as expressed in the example above.

Alternatives Considered

• Solution variants expressed above
• Add ochami CLI nodes file-formatted output to Magellan: This would work, but since Magellan is meant to be a generic tool, it may make more architectural sense to adapt the OpenCHAMI-specific tool (ochami CLI) to the generic tool (Magellan)
• Keep current implementations: this would obviously require more manual work to adapt Magellan output to an ochami nodes file format, as expressed above

Other Considerations

Introduced code maintenance, etc.

Related Docs / PRs

No response

[alexlovelltroy]

Our nodes.yaml file was originally a stopgap solution until magellan was ready to properly populate smd. It has gained associated tooling, but was never "deisgned" to meet needs beyond bootstrapping a cluster.

If we want to create more tooling around it, I want to start by formalizing it with a well-documented schema and set of tooling for linting and testing. We might even want to include it as a first class resource in the inventory-service for bulk loading a set of nodes at once.

I'm opposed to adding more tooling around nodes.yaml without formalizing it. If we proceed, I'm more interested in Solution 1, but I'd like to back up and consider the problem we're trying to solve with this feature request and if there are other ways of accomplishing the goal that might be more sustainable in the long term.

[davidallendj]

I agree with formalizing the structure. The original goal in magellan was to match the format so the inventory data could be sent to SMD to populate. If we formalize on the structure, we can just change the output format in magellan and both should be able to submit without having to add anymore code ideally. I'm thinking Solution 1 might not even be necessary if we can do magellan collect | ochami discovery dynamic or something like that instead of the conversion step.

[synackd]

I'm opposed to adding more tooling around nodes.yaml without formalizing it.

If we want to create more tooling around it, I want to start by formalizing it with a well-documented schema and set of tooling for linting and testing. We might even want to include it as a first class resource in the inventory-service for bulk loading a set of nodes at once.

Good points! I agree with these as well. Now that we are writing our own services instead of relying on the CSM codebase, it would be good to revisit how we want to formalize discovery. I think supporting it as a first-class citizen in the inventory-service would be prudent, perhaps with a potential node abstraction (though that's more relevant to OpenCHAMI/roadmap#134 and not here).

It sounds like before we decide on a solution, we'll want to formalize what we want discovery to look like. Does that sound right? If so, is the advised next step to create an RFD to hash out what that might look like?

[davidallendj]

It sounds like before we decide on a solution, we'll want to formalize what we want discovery to look like. Does that sound right? If so, is the advised next step to create an RFD to hash out what that might look like?

That sounds right to me. I would think we would need to understand the abstractions around inventory-service too to really be able to hash that out so a RFD seems appropriate IMO.