diff --git a/packs/stormforge-agent-2.28.5/README.md b/packs/stormforge-agent-2.28.5/README.md new file mode 100644 index 00000000..08528153 --- /dev/null +++ b/packs/stormforge-agent-2.28.5/README.md @@ -0,0 +1,125 @@ +# StormForge Agent + +## Purpose + +The StormForge Agent continuously observes workload behaviour across your Kubernetes +cluster and applies ML-driven resource recommendations (CPU/memory requests and limits) +to reduce waste and improve reliability. It consists of two components: + +| Component | Role | +|-----------|------| +| **workload-agent** | Watches workloads, fetches recommendations from the StormForge API, and applies patches | +| **metrics-forwarder** | Lightweight Prometheus agent that scrapes cAdvisor & kube-state-metrics and remote-writes to StormForge | + +--- + +## Prerequisites + +| Requirement | Details | +|-------------|---------| +| Kubernetes | ≥ 1.29 | +| Helm | ≥ 3.8 (OCI support required) | +| Namespace | `stormforge-system` (created automatically by the chart) | +| StormForge account | Sign up at | +| OAuth2 credentials | `CLIENT_ID` and `CLIENT_SECRET` from **StormForge → Settings → API Access** or via `stormforge create auth` | +| Outbound HTTPS | Egress to `*.stormforge.io:443` and `in.stormforge.io:443` | + +--- + +## Key `values.yaml` Parameters + +### Required + +| Parameter | Description | +|-----------|-------------| +| `clusterName` | Unique name for this cluster in the StormForge platform | +| `authorization.clientID` | OAuth2 client ID | +| `authorization.clientSecret` | OAuth2 client secret — use an ExternalSecret or SealedSecret | + +### Workload Controller + +| Parameter | Default | Description | +|-----------|---------|-------------| +| `workload.image.repository` | `registry.stormforge.io/optimize/workload-agent` | Image repository | +| `workload.image.tag` | `2.28.5` | Image tag | +| `workload.resources` | 20m CPU / 64Mi RAM | Resource requests | +| `workload.workloadSyncInterval` | `50m` | Heartbeat interval for active workloads | +| `workload.collectLabels` | `true` | Include workload labels in API sync | +| `workload.inactiveWorkloadsGC` | `true` | Garbage-collect stale workloads on the API | + +### Metrics Forwarder (Prometheus) + +| Parameter | Default | Description | +|-----------|---------|-------------| +| `prom.image.repository` | `quay.io/prometheus/prometheus` | Image repository (swap to `prom/prometheus` for Docker Hub) | +| `prom.image.tag` | `v3.13.1` | Prometheus version | +| `prom.resources` | 20m CPU / 64Mi RAM | Resource requests | +| `prom.scrapeInterval` | `30s` | Scrape cadence | +| `prom.remoteWriteUrl` | `https://in.stormforge.io/prometheus/write` | Remote-write endpoint | +| `storageVolumeSize` | `2G` | WAL volume size | + +### Network / Proxy + +| Parameter | Default | Description | +|-----------|---------|-------------| +| `proxyUrl` | `""` | HTTPS proxy for egress to StormForge (e.g. `https://proxy.corp.example.com/`) | +| `noProxy` | RFC-1918 ranges | Comma-separated list of addresses to bypass the proxy | + +### Platform Flags + +| Parameter | Default | Description | +|-----------|---------|-------------| +| `openshift` | `false` | Enable OpenShift compatibility | +| `gkeAutopilot` | `false` | Enable GKE Autopilot compatibility | +| `enableCostMetrics` | `false` | Collect cost-related node metrics | +| `nodeCPUPressureDetection` | `false` | Detect node CPU pressure via PSI (requires Linux ≥ 4.20, cgroup v2) | + +### Feature Gates + +| Parameter | Default | Description | +|-----------|---------|-------------| +| `featureGates.optimizeCronJobs` | `false` | CronJob optimization support | +| `featureGates.optimizeSparkOperator` | `false` | Spark Operator support | +| `featureGates.optimizeGithubARC` | `false` | GitHub Actions Runner Controller support | + +--- + +## Minimal Configuration Example + +```yaml +pack: + namespace: stormforge-system + +charts: + stormforge-agent: + clusterName: "my-production-cluster" + authorization: + clientID: "{{ .spectrocontext.CLIENT_ID }}" + clientSecret: "{{ .spectrocontext.CLIENT_SECRET }}" +``` + +> **Security note:** Never commit `clientSecret` in plain text. +> secret management or reference a Kubernetes secret via `manageAuthSecret: false` +> and provision the secret externally. + +--- + +## Disable Agent-Managed Auth Secret + +If you prefer to manage the auth secret externally (e.g. with Vault or External Secrets Operator): + +```yaml +charts: + stormforge-agent: + manageAuthSecret: false + authSecret: "my-external-auth-secret" # must exist in the target namespace +``` + +--- + +## Useful Links + +- Documentation: +- StormForge web app: +- Helm chart source: `oci://registry.stormforge.io/library/stormforge-agent` +- Changelog: diff --git a/packs/stormforge-agent-2.28.5/charts/stormforge-agent-2.28.5.tgz b/packs/stormforge-agent-2.28.5/charts/stormforge-agent-2.28.5.tgz new file mode 100644 index 00000000..6cc03c44 Binary files /dev/null and b/packs/stormforge-agent-2.28.5/charts/stormforge-agent-2.28.5.tgz differ diff --git a/packs/stormforge-agent-2.28.5/charts/stormforge-agent/Chart.yaml b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/Chart.yaml new file mode 100644 index 00000000..d9c04c43 --- /dev/null +++ b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/Chart.yaml @@ -0,0 +1,12 @@ +apiVersion: v2 +appVersion: 2.28.5 +description: Bundled StormForge Agent and Prometheus in Agent Mode capturing telemetry + for StormForge systems. +home: https://www.stormforge.io/ +icon: https://app.stormforge.io/img/logo.png +kubeVersion: '>= 1.16.x-0' +maintainers: +- name: grambot +name: stormforge-agent +type: application +version: 2.28.5 diff --git a/packs/stormforge-agent-2.28.5/charts/stormforge-agent/README.md b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/README.md new file mode 100644 index 00000000..8d219825 --- /dev/null +++ b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/README.md @@ -0,0 +1,279 @@ +# StormForge Agent Helm Installation + +The StormForge Agent is a helm chart which combines the stormforge-agent that surfaces minimum Kubernetes resource (pods, hpa) metrics and a prometheus agent to forward these metrics to StormForge backend. + +## Installation + +Generating the credentials: + +``` +stormforge auth create AUTH_NAME +``` + +This command will generate the following file. Save the file locally, i.e. as `AUTH_NAME-credentials.yaml`: + +``` +stormforge: + address: https://api.stormforge.io/ +authorization: + issuer: https://api.stormforge.io/ + clientID: # AUTH_NAME + clientSecret: +``` + +Running the installation (replace `LATEST_VERSION` and `CLUSTER_NAME` in example with appropriate values) + +``` +helm install stormforge-agent oci://registry.stormforge.io/library/stormforge-agent \ + --version LATEST_VERSION \ + --namespace stormforge-system \ + --create-namespace \ + --values AUTH_NAME-credentials.yaml \ + --set clusterName=CLUSTER_NAME +``` + +## Upgrade + +A typical upgrade might look something like the following. + +``` +helm upgrade stormforge-agent oci://registry.stormforge.io/library/stormforge-agent \ + --version LATEST_VERSION \ + --namespace stormforge-system \ + --values AUTH_NAME-credentials.yaml \ + --set clusterName=CLUSTER_NAME +``` + +If you no longer have access to the Chart values used to install the agent but don't need to change any of them, you might be able to retrive and reuse the existing values using `helm get values` as follows. + +``` +helm get values -n stormforge-system stormforge-agent -o yaml > existing-stormforge-agent-values.yaml +helm upgrade stormforge-agent oci://registry.stormforge.io/library/stormforge-agent \ + --version LATEST_VERSION \ + --namespace stormforge-system \ + --values existing-stormforge-agent-values.yaml +``` + +> Note: It is strongly suggested that you **do not** use Helm's `--reuse-values` flag when upgrading to new versions. This flag can result in default Chart values defined in an old Chart version being incorrectly carried forward and used in the new one, compromising the configuration in unpredictable ways. + +## Helm Config + +Please look at the charts/stormforge-agent/values.yaml file for Helm configuration options. + +``` +helm show values oci://registry.stormforge.io/library/stormforge-agent --version LATEST_VERSION +``` + +### Configuration + +The following Helm configuration values address most common scenarios. Additional configuration parameters exist. For a listing of all available values, see the [values.yaml](values.yaml) file. + +#### Required Parameters: + +- **`clusterName`**: value is `string`. Set from helm installation. +- **`authorization`**: value is `dict`. This value will be supplied by `stormforge auth create` or obtained from the StormForge app. + +#### Optional Parameters: + +- **`proxyUrl`**: value is `string`. Set to configure a proxy used to access `stormforge.io` services. Example: `https://proxy.example.com`. +- **`noProxy`**: value is `string`. If using a proxy, this value should at a minimum be set to the subnet(s) for the cluster. Example: `10.76.0.0/16`. +- **`workload.watchWorkloadResources`**: value is `boolean`. Default is `true`. Agent proactively identifies workloads with zero replicas. + If set, workload controller will watch create/delete pod events and derive StormForge workloads from them. When `false`, the only way to optimize workloads is by using WorkloadOptimizer resources. +- **`workload.allowNamespaces`**: values are `[]string`. If set, only these namespaces will be visible for workload discovery. +- **`workload.denyNamespaces`**: values are `[]string`. If set, all namespaces but these will be visible for workload discovery. + +Attention: `workload.allowNamespaces` and `workload.denyNamespaces` are mutually exclusive, with `workload.allowNamespaces` taking precedence. + +The namespace `"kube-system"` is always excluded from the workload discovery (unless set by `allowNamespaces`). + +## Workload Metrics + +Here are the workload metrics we produce. + +| Metric | Source | Why | +| ------------------------------------------------ | ----------------------------------- | ----------------------------------------------------------------------------- | +| sf_workload_pod_owner | Consolidated metric for ownership | With this metric, we have pod owner and workload, replacing KSM kube_pod_owner and kube_replicaset_owner | +| sf_workload_spec_replicas | Consolidated metric for desired replicas number | With this metric, we have all desired replica metrics regardless type of pod owner. The pod owner must have the subresource scale. | +| sf_workload_status_replicas | Consolidated metric for observed replicas number | With this metric, we have all observed replica metrics regardless type of pod owner. | +| sf_workload_pod_container_resource_requests | Consolidated pod metric with requests | With this metric, we have all requests metrics in a single metric. | +| sf_workload_pod_container_resource_limits | Consolidated pod metric with limits | With this metric, we have all limits metrics in a single metric. | +| sf_workload_terminated_count | Consolidated metric for workload termination events | With this metric, we track the count of times a workload was terminated (e.g. OOMKilled) | +| sf_horizontalpodautoscaler_spec_min_replicas | KSM-like/horizontalpodautoscaler-metrics | Track minimum replicas for each HPA | +| sf_horizontalpodautoscaler_spec_max_replicas | KSM-like/horizontalpodautoscaler-metrics | Track maximum replicas for each HPA | +| sf_horizontalpodautoscaler_spec_target_metric | KSM-like/horizontalpodautoscaler-metrics | Track target metric for each HPA | +| sf_node_allocated_requests | Custom node metrics | The number of allocated requests on the node. | +| sf_node_allocated_limits | Custom node metrics | The number of allocated limits on the node. | +| sf_node_allocated_pods | Custom node metrics | The number of non terminated pods running on the node. | +| sf_node_allocatable_resources | Custom node metrics | The amount of resources available to be allocated on the node | +| sf_node_capacity_resources | Custom node metrics | The total amount of resources that a node has. Used in calculating the average cluster CPU and memory utilization. | +| container_cpu_usage_seconds_total | cadvisor | Track cpu usage for each container | +| container_memory_working_set_bytes | cadvisor | Track memory usage for each container | +| container_cpu_cfs_throttled_seconds_total | cadvisor | Total time duration the container has been throttled | +| container_cpu_cfs_throttled_periods_total | cadvisor | Number of throttled period intervals | +| container_cpu_cfs_periods_total | cadvisor | Number of elapsed enforcement period intervals | +| container_memory_max_usage_bytes | cadvisor | Maximum memory usage recorded | +| node_cpu_usage_seconds_total | Resource metrics | Used in calculating the average cluster CPU utilization | +| node_memory_working_set_bytes | Resource metrics | Used in calculating the average cluster memory utilization | + +Individual tenants could have additional metrics. + +### Extra GO / Controller-Runtime Metrics + +Starting on version `2.20.1`, by default, we have started to forward to our backend all GO and controller-runtime metrics. +These are generated by the runtime, and it is outside the scope document them in detail. + +Here are the metrics: + +``` +certwatcher_read_certificate_errors_total +certwatcher_read_certificate_total +controller_runtime_active_workers +controller_runtime_max_concurrent_reconciles +controller_runtime_reconcile_errors_total +controller_runtime_reconcile_panics_total +controller_runtime_reconcile_time_seconds_bucket +controller_runtime_reconcile_time_seconds_count +controller_runtime_reconcile_time_seconds_sum +controller_runtime_reconcile_total +controller_runtime_terminal_reconcile_errors_total +controller_runtime_webhook_panics_total +go_gc_duration_seconds +go_gc_duration_seconds_count +go_gc_duration_seconds_sum +go_gc_gogc_percent +go_gc_gomemlimit_bytes +go_goroutines +go_info +go_memstats_alloc_bytes +go_memstats_alloc_bytes_total +go_memstats_buck_hash_sys_bytes +go_memstats_frees_total +go_memstats_gc_sys_bytes +go_memstats_heap_alloc_bytes +go_memstats_heap_idle_bytes +go_memstats_heap_inuse_bytes +go_memstats_heap_objects +go_memstats_heap_released_bytes +go_memstats_heap_sys_bytes +go_memstats_last_gc_time_seconds +go_memstats_mallocs_total +go_memstats_mcache_inuse_bytes +go_memstats_mcache_sys_bytes +go_memstats_mspan_inuse_bytes +go_memstats_mspan_sys_bytes +go_memstats_next_gc_bytes +go_memstats_other_sys_bytes +go_memstats_stack_inuse_bytes +go_memstats_stack_sys_bytes +go_memstats_sys_bytes +go_sched_gomaxprocs_threads +go_threads +rest_client_requests_total +workqueue_adds_total +workqueue_depth +workqueue_longest_running_processor_seconds +workqueue_queue_duration_seconds_bucket +workqueue_queue_duration_seconds_count +workqueue_queue_duration_seconds_sum +workqueue_retries_total +workqueue_unfinished_work_seconds +workqueue_work_duration_seconds_bucket +workqueue_work_duration_seconds_count +workqueue_work_duration_seconds_sum +``` + +## Troubleshooting StormForge Agent + +### Getting Logs from Prometheus Agent + +In case one does not see data on AMP, check the prometheus agent logs. In this example below, the agent is running on namespace `stormforge-system`. Please be aware there are two deployments, one for the prometheus agent and another for the stormforge workload controller: + +```sh +# workload controller +kubectl logs -l app.kubernetes.io/component=agent --tail=-1 -n stormforge-system + +# prometheus agent +kubectl logs -l app.kubernetes.io/component=metrics-forwarder --tail=-1 -n stormforge-system + +``` + +If there is no errors, see the next steps. + +### Verify Prometheus Targets + +When you install the agent, you should be sure to verify it is actually able to scrape the workload metrics. In particular, stormforge-agent has a static url config which makes it config error prone, which is `https://<>:8080/metrics`. In this example below, the agent is running on namespace `stormforge-system` + +```sh +# e.g. +kubectl expose deploy/stormforge-agent-metrics-forwarder -n stormforge-system +kubectl port-forward deploy/stormforge-agent-metrics-forwarder 9090:9090 -n stormforge-system +# http://localhost:9090/targets?search= to validate targets are being collected +``` + +To look at the actual metrics from the perspective of the stormforge-agent controller: + +```sh +# e.g. +kubectl expose deploy/stormforge-agent-workload-controller -n stormforge-system +kubectl port-forward deploy/stormforge-agent-workload-controller 8080:8080 -n stormforge-system +# http://localhost:8080/metrics to validate targets are being collected +``` + +### Checking Prometheus WAL + +Data should be on the WAL. In this example below, the agent is running on namespace `stormforge-system`: + +```sh +# e.g. +kubectl exec $(kubectl get pods -n stormforge-system -l app.kubernetes.io/component=metrics-forwarder | grep -v NAME | awk '{print $1}') -n stormforge-system -it -c prometheus -- sh + +# inside the pod +$ promtool tsdb dump data-agent/ | head + +# check sf workload metrics +$ promtool tsdb dump data-agent/ | grep sf_workload | head -5 + +# check horizontal metrics +$ promtool tsdb dump data-agent/ | grep horizontal | head -5 + +``` + +By default, we are holding 30 minutes on data on WAL. + +### Credentials + +Credentials are not authorized, ask permission: + +``` +# kubectl logs --tail=-1 -n stormforge-system -l app.kubernetes.io/component=metrics-forwarder -c prometheus + +ts=2023-02-13T22:21:55.813Z caller=dedupe.go:112 component=remote level=error remote_name=d24ad1 url=https://in.dev-1.dev.gramlabs.dev/prometheus/write msg="non-recoverable error" count=77 exemplarCount=0 err="server returned HTTP status 404 Not Found: {\"message\":null}" +``` + +Bad credentials, double check parameters passed during installation (i.e. secrets): + +``` +# kubectl logs --tail=-1 -n stormforge-system -l app.kubernetes.io/component=metrics-forwarder -c prometheus + +ts=2023-02-13T22:25:48.460Z caller=dedupe.go:112 component=remote level=error remote_name=0745da url=https://in.dev-1.dev.gramlabs.dev/prometheus/write msg="non-recoverable error" count=35 exemplarCount=0 err="server returned HTTP status 401 Unauthorized: Authorization malformed or invalid" +ts=2023-02-13T22:26:03.506Z caller=dedupe.go:112 component=remote level=error remote_name=0745da url=https://in.dev-1.dev.gramlabs.dev/prometheus/write msg="non-recoverable error" count=77 exemplarCount=0 err="server returned HTTP status 401 Unauthorized: Authorization malformed or invalid" +``` + +### Enable debug logging + +Debug logging can now be enabled via http requests. +This should make it more useful to enable debug logging for a short period. + +The default log level is `1` ( info ). + +This can be changed by: +``` +kubectl port-forward -n stormforge-system 6060:6060 + +# Default info level logging +curl -X PUT localhost:6060/debug/loglevel -d level=1 +# Verbose/Debug logging +curl -X PUT localhost:6060/debug/loglevel -d level=5 +# Trace logging +curl -X PUT localhost:6060/debug/loglevel -d level=9 +``` diff --git a/packs/stormforge-agent-2.28.5/charts/stormforge-agent/aws_mp_configuration_schema.json b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/aws_mp_configuration_schema.json new file mode 100644 index 00000000..f20e8ad8 --- /dev/null +++ b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/aws_mp_configuration_schema.json @@ -0,0 +1,155 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema", + "required": [ + "standbyMode", + "authorization" + ], + "properties": { + "standbyMode": { + "type": "boolean" + }, + "authorization": { + "type": "object", + "properties": { + "clientID": { + "type": "string", + "pattern": "^[^\n]*$" + }, + "clientSecret": { + "type": "string", + "pattern": "^[^\n]*$" + }, + "issuer": { + "type": "string", + "pattern": "^[^\n]*$" + } + }, + "additionalProperties": false + }, + "proxyUrl": { + "type": "string", + "description": "Configuration for HTTPS_PROXY environment variable" + }, + "noProxy": { + "type": "string", + "description": "Configuration for NO_PROXY environment variable" + }, + "manageAuthSecret": { + "type": "boolean", + "description": "Whether or not to create and manage the authorization secret" + }, + "workload": { + "type": "object", + "properties": { + "workloadSyncInterval": { + "type": "string", + "description": "Interval the agent will update probeTime for each existent workload." + }, + "collectLabels": { + "type": "boolean", + "description": "Collect labels of the cluster's workloads." + }, + "inactiveWorkloadsGC": { + "type": "boolean", + "description": "Garbage collection of inactive workloads on StormForge API backend." + }, + "denyNamespaces": { + "type": ["array", "null"], + "description": "A list of namespaces (regex) to ignore workloads and metrics from", + "items": { + "type": "string" + } + }, + "allowNamespaces": { + "type": ["array", "null"], + "description": "A list of namespaces (regex) to include workloads and metrics from", + "items": { + "type": "string" + } + } + } + }, + "prom": { + "type": "object", + "properties": { + "proxy": { "type": "object" }, + "remoteWriteUrl": { "type": "string" }, + "resources": { "type": "object" }, + "debug": { "type": "boolean" }, + "scrapeInterval": { "type": "string" }, + "scrapeTimeout": { "type": "string" }, + "remoteTimeout": { "type": "string" }, + "retentionMaxTime": { "type": "string" }, + "retentionMinTime": { "type": "string" }, + "tokenUrl": { "type": "string" }, + "remoteWriteQueueConfig": { "type": "object" }, + "honorTimestamps": { "type": "boolean" }, + "extraCAdvisorMetrics": { + "type": "array", + "items": { + "type": "string" + } + } + }, + "additionalProperties": false + }, + "clusterDefaultConfig": { + "type": "object", + "propertyNames": { + "enum": [ + "schedule", + "autoDeploy", + "cpuOptimizationGoal", + "memoryOptimizationGoal", + "containersCpuOptimizationPolicy", + "containersCpuRequestsMin", + "containersCpuRequestsMax", + "containersCpuLimitsMin", + "containersCpuLimitsMax", + "containersCpuLimitsLimitRequestRatio", + "containersMemoryOptimizationPolicy", + "containersMemoryRequestsMin", + "containersMemoryRequestsMax", + "containersMemoryLimitsMin", + "containersMemoryLimitsMax", + "containersMemoryLimitsLimitRequestRatio" + ] + }, + "properties": { + "schedule": { + "type": "string", + "pattern": "(^([-/(),\\*\\w]+ ){4,5}[-/(),\\*\\w]+$)|(^P\\w+$)|(^@\\w+$)" + }, + "autoDeploy": { + "type": ["string", "boolean"], + "enum": ["true", true, "false", false] + } + }, + "patternProperties": { + "^(cpu|memory)OptimizationGoal$": { + "enum": ["balanced", "savings", "reliability"] + }, + "^containers": { + "pattern": "^([-.A-Za-z0-9]*[,=]?)+$" + } + } + }, + "extraEnvVars": { + "type": "array", + "items": { + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + }, + "additionalProperties": false + }, + "x-kubernetes-patch-merge-key": "name", + "x-kubernetes-patch-strategy": "merge" + } + }, + "additionalProperties": true +} diff --git a/packs/stormforge-agent-2.28.5/charts/stormforge-agent/ci/ci-values.yaml b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/ci/ci-values.yaml new file mode 100644 index 00000000..a5595ef3 --- /dev/null +++ b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/ci/ci-values.yaml @@ -0,0 +1,8 @@ +--- +clusterName: my-cluster +stormforge: + address: https://api.stormforge.io/ +authorization: + clientID: "id" + clientSecret: "secret" + issuer: "https://api.stormforge.io/" diff --git a/packs/stormforge-agent-2.28.5/charts/stormforge-agent/crds/optimize.stormforge.io_clusteroptimizationconfigurations.yaml b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/crds/optimize.stormforge.io_clusteroptimizationconfigurations.yaml new file mode 100644 index 00000000..7d36379e --- /dev/null +++ b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/crds/optimize.stormforge.io_clusteroptimizationconfigurations.yaml @@ -0,0 +1,206 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.19.0 + name: clusteroptimizationconfigurations.optimize.stormforge.io +spec: + group: optimize.stormforge.io + names: + kind: ClusterOptimizationConfiguration + listKind: ClusterOptimizationConfigurationList + plural: clusteroptimizationconfigurations + shortNames: + - coc + singular: clusteroptimizationconfiguration + scope: Cluster + versions: + - name: v1 + schema: + openAPIV3Schema: + description: ClusterOptimizationConfiguration is the Schema for the ClusterOptimizationConfiguration + API. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: ClusterOptimizationConfigurationSpec defines the desired + state of ClusterOptimizationConfiguration. + properties: + order: + default: 0 + type: integer + x-kubernetes-validations: + - message: value must be an integer between -1000 and 1000 + rule: self >= -1000 && self <= 1000 + rules: + items: + description: ClusterOptimizationConfig defines a single optimization + configuration entry. + properties: + matchConditions: + items: + description: OptimizationMatchCondition defines a single match + condition using CEL expressions. + properties: + expression: + type: string + name: + type: string + required: + - expression + - name + type: object + type: array + name: + type: string + namespaceSelector: + description: |- + A label selector is a label query over a set of resources. The result of matchLabels and + matchExpressions are ANDed. An empty label selector matches all objects. A null + label selector matches no objects. + properties: + matchExpressions: + description: matchExpressions is a list of label selector + requirements. The requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key that the selector + applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + selector: + description: |- + A label selector is a label query over a set of resources. The result of matchLabels and + matchExpressions are ANDed. An empty label selector matches all objects. A null + label selector matches no objects. + properties: + matchExpressions: + description: matchExpressions is a list of label selector + requirements. The requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key that the selector + applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + settings: + items: + description: ConfigItem represents a single configuration + entry with a name and one or more values. + properties: + name: + type: string + value: + type: string + values: + items: + type: string + type: array + required: + - name + type: object + x-kubernetes-validations: + - message: either value or values must be set, but not both + rule: (has(self.value) && !has(self.values)) || (!has(self.value) + && has(self.values)) + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + required: + - name + - settings + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + type: object + type: object + served: true + storage: true diff --git a/packs/stormforge-agent-2.28.5/charts/stormforge-agent/crds/optimize.stormforge.io_optimizationconfigurations.yaml b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/crds/optimize.stormforge.io_optimizationconfigurations.yaml new file mode 100644 index 00000000..9df9d6d2 --- /dev/null +++ b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/crds/optimize.stormforge.io_optimizationconfigurations.yaml @@ -0,0 +1,157 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.19.0 + name: optimizationconfigurations.optimize.stormforge.io +spec: + group: optimize.stormforge.io + names: + kind: OptimizationConfiguration + listKind: OptimizationConfigurationList + plural: optimizationconfigurations + shortNames: + - oc + singular: optimizationconfiguration + scope: Namespaced + versions: + - name: v1 + schema: + openAPIV3Schema: + description: OptimizationConfiguration is the Schema for the OptimizationConfiguration + API. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: OptimizationConfigurationSpec defines the desired state of + OptimizationConfiguration. + properties: + order: + default: 0 + type: integer + x-kubernetes-validations: + - message: value must be an integer between -1000 and 1000 + rule: self >= -1000 && self <= 1000 + rules: + items: + description: OptimizationConfig defines a single optimization configuration + entry. + properties: + matchConditions: + items: + description: OptimizationMatchCondition defines a single match + condition using CEL expressions. + properties: + expression: + type: string + name: + type: string + required: + - expression + - name + type: object + type: array + name: + type: string + selector: + description: |- + A label selector is a label query over a set of resources. The result of matchLabels and + matchExpressions are ANDed. An empty label selector matches all objects. A null + label selector matches no objects. + properties: + matchExpressions: + description: matchExpressions is a list of label selector + requirements. The requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key that the selector + applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + settings: + items: + description: ConfigItem represents a single configuration + entry with a name and one or more values. + properties: + name: + type: string + value: + type: string + values: + items: + type: string + type: array + required: + - name + type: object + x-kubernetes-validations: + - message: either value or values must be set, but not both + rule: (has(self.value) && !has(self.values)) || (!has(self.value) + && has(self.values)) + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + required: + - name + - settings + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + type: object + type: object + served: true + storage: true diff --git a/packs/stormforge-agent-2.28.5/charts/stormforge-agent/crds/optimize.stormforge.io_workloadoptimizers.yaml b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/crds/optimize.stormforge.io_workloadoptimizers.yaml new file mode 100644 index 00000000..0fef3595 --- /dev/null +++ b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/crds/optimize.stormforge.io_workloadoptimizers.yaml @@ -0,0 +1,736 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.19.0 + name: workloadoptimizers.optimize.stormforge.io +spec: + group: optimize.stormforge.io + names: + kind: WorkloadOptimizer + listKind: WorkloadOptimizerList + plural: workloadoptimizers + shortNames: + - wo + singular: workloadoptimizer + scope: Namespaced + versions: + - additionalPrinterColumns: + - description: WorkloadOptimizer state + jsonPath: .status.state + name: State + type: string + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + name: v1 + schema: + openAPIV3Schema: + description: WorkloadOptimizer is the Schema for the WorkloadOptimizer API. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + properties: + apply: + description: |- + Apply Method is used to configure how Optimize Live deploys the complete recommendations + that are generated when the workload’s learning period is over. + properties: + maxPercentDecrease: + description: MaxPercentDecrease limits the maximum percent decrease + applied per deploy. + type: string + maxPercentIncrease: + description: MaxPercentIncrease limits the maximum percent increase + applied per deploy. + type: string + method: + enum: + - patchworkloadresources + - PatchWorkloadResources + - dynamicadmissionwebhook + - DynamicAdmissionWebhook + - inplacepodresizing + - InPlacePodResizing + type: string + type: object + autoDeploy: + type: boolean + containerSettings: + items: + description: |- + Container is used to configure the workload container settings. + This should allow for the ability to set container specific resource bounds. + properties: + cpu: + description: ResourceSettings configures the specific requests + and limits for the container. + properties: + limits: + description: LimitsSettings configures the container limits. + properties: + limitRequestRatio: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + max: + anyOf: + - type: integer + - type: string + description: |- + Max specifies the upper bound of the recommendation. + If a recommendation is produced outside of the defined bound it will be clipped. + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + min: + anyOf: + - type: integer + - type: string + description: |- + Min specifies the lower bound of the recommendation. + If a recommendation is produced outside of the defined bound it will be clipped. + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + patchFormat: + description: PatchFormat specifies a Go template used + to format the value for the patched resource. + type: string + patchPath: + description: |- + PatchPath controls where the recommendation should be applied/patched. + The format for patchPath is yamlPath format. + type: string + type: object + optimizationPolicy: + description: Policy dictates the OptimizationPolicy that + should be used. + enum: + - requestsandlimits + - RequestsAndLimits + - requestsonly + - RequestsOnly + - donotoptimize + - DoNotOptimize + - requestsraiselimitsifneeded + - RequestsRaiseLimitsIfNeeded + type: string + requests: + description: RequestsSettings configures the container requests. + properties: + max: + anyOf: + - type: integer + - type: string + description: |- + Max specifies the upper bound of the recommendation. + If a recommendation is produced outside of the defined bound it will be clipped. + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + min: + anyOf: + - type: integer + - type: string + description: |- + Min specifies the lower bound of the recommendation. + If a recommendation is produced outside of the defined bound it will be clipped. + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + patchFormat: + description: PatchFormat specifies a Go template used + to format the value for the patched resource. + type: string + patchPath: + description: |- + PatchPath controls where the recommendation should be applied/patched. + The format for patchPath is yamlPath format. + type: string + type: object + type: object + memory: + description: ResourceSettings configures the specific requests + and limits for the container. + properties: + limits: + description: LimitsSettings configures the container limits. + properties: + limitRequestRatio: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + max: + anyOf: + - type: integer + - type: string + description: |- + Max specifies the upper bound of the recommendation. + If a recommendation is produced outside of the defined bound it will be clipped. + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + min: + anyOf: + - type: integer + - type: string + description: |- + Min specifies the lower bound of the recommendation. + If a recommendation is produced outside of the defined bound it will be clipped. + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + patchFormat: + description: PatchFormat specifies a Go template used + to format the value for the patched resource. + type: string + patchPath: + description: |- + PatchPath controls where the recommendation should be applied/patched. + The format for patchPath is yamlPath format. + type: string + type: object + optimizationPolicy: + description: Policy dictates the OptimizationPolicy that + should be used. + enum: + - requestsandlimits + - RequestsAndLimits + - requestsonly + - RequestsOnly + - donotoptimize + - DoNotOptimize + - requestsraiselimitsifneeded + - RequestsRaiseLimitsIfNeeded + type: string + requests: + description: RequestsSettings configures the container requests. + properties: + max: + anyOf: + - type: integer + - type: string + description: |- + Max specifies the upper bound of the recommendation. + If a recommendation is produced outside of the defined bound it will be clipped. + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + min: + anyOf: + - type: integer + - type: string + description: |- + Min specifies the lower bound of the recommendation. + If a recommendation is produced outside of the defined bound it will be clipped. + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + patchFormat: + description: PatchFormat specifies a Go template used + to format the value for the patched resource. + type: string + patchPath: + description: |- + PatchPath controls where the recommendation should be applied/patched. + The format for patchPath is yamlPath format. + type: string + type: object + type: object + name: + type: string + required: + - name + type: object + type: array + hpaSettings: + description: Autoscaler handles the configuration for the HPA configured + for the workload. + properties: + metrics: + description: Metrics configures the HPA metrics the workload scales + on. + items: + description: AutoscalerMetrics holds the configuration for HPA + metrics. + properties: + containerResource: + description: Configuration for a Container metric based + HPA metric type. + properties: + container: + type: string + name: + type: string + type: object + external: + description: Configuration for an External metric based + HPA metric type. + properties: + name: + description: name is the name of the given metric + type: string + selector: + description: |- + selector is the string-encoded form of a standard kubernetes label selector for the given metric + When set, it is passed as an additional parameter to the metrics server for more specific metrics scoping. + When unset, just the metricName will be used to gather metrics. + properties: + matchExpressions: + description: matchExpressions is a list of label + selector requirements. The requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key that the + selector applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + required: + - name + type: object + object: + description: Configuration for an Object metric based HPA + metric type. + properties: + describedObject: + description: TargetRef specifies the kind and name of + the workload object. + properties: + apiVersion: + description: API version of the referent + type: string + kind: + description: |- + Kind of the referent; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + Restricted to [deployments|statefulsets|daemonsets|rollouts] for initial release + enum: + - deployments + - statefulsets + - daemonsets + - rollouts + type: string + name: + description: 'Name of the referent; More info: http://kubernetes.io/docs/user-guide/identifiers#names' + type: string + required: + - kind + - name + type: object + name: + description: name is the name of the given metric + type: string + selector: + description: |- + selector is the string-encoded form of a standard kubernetes label selector for the given metric + When set, it is passed as an additional parameter to the metrics server for more specific metrics scoping. + When unset, just the metricName will be used to gather metrics. + properties: + matchExpressions: + description: matchExpressions is a list of label + selector requirements. The requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key that the + selector applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + required: + - name + type: object + pods: + description: Configuration for a Pod metric based HPA metric + type. + properties: + name: + description: name is the name of the given metric + type: string + selector: + description: |- + selector is the string-encoded form of a standard kubernetes label selector for the given metric + When set, it is passed as an additional parameter to the metrics server for more specific metrics scoping. + When unset, just the metricName will be used to gather metrics. + properties: + matchExpressions: + description: matchExpressions is a list of label + selector requirements. The requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key that the + selector applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + required: + - name + type: object + resource: + description: Configuration for a Resource metric based HPA + metric type. + properties: + name: + type: string + type: object + target: + description: Target defines the bounds for the recommended + value. + properties: + max: + type: integer + min: + type: integer + patchPath: + type: string + type: + description: |- + MetricTargetType specifies the type of metric being targeted, and should be either + "Value", "AverageValue", or "Utilization" + type: string + type: object + type: object + type: array + patchTargetRef: + description: |- + PatchTargetRef allows specifiying the object that should be patched + with HPA settings. + This is most useful in situations where a CRD is used to configure a HPA. + properties: + apiVersion: + type: string + kind: + description: 'Kind of the referent; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + name: + description: 'Name of the referent; More info: http://kubernetes.io/docs/user-guide/identifiers#names' + type: string + required: + - kind + - name + type: object + type: object + learningPeriod: + type: string + patchTargetRef: + description: PatchTargetRef specifies the kind and name of the object + that should be patched. + properties: + apiVersion: + type: string + kind: + description: 'Kind of the referent; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + name: + description: 'Name of the referent; More info: http://kubernetes.io/docs/user-guide/identifiers#names' + type: string + required: + - kind + - name + type: object + podScheduling: + description: PodScheduling handles the configuration for the behavior + of optimizing pod scheduling. + properties: + instanceCategories: + description: SchedulingInstanceCategories describes optimization + for instance category affinities. + properties: + nodeAffinity: + properties: + type: + enum: + - soft + - Soft + - preferred + - Preferred + - preferredDuringSchedulingIgnoredDuringExecution + - preferredduringschedulingignoredduringexecution + - hard + - Hard + - required + - Required + - requiredDuringSchedulingIgnoredDuringExecution + - requiredduringschedulingignoredduringexecution + type: string + weight: + type: integer + type: object + patchFormat: + type: string + patchPath: + type: string + type: object + schedulingOptimizationPolicy: + description: PodSchedulingOptimizationPolicy is used to configure + whether to optimize NodeAffinity. + enum: + - instancecategories + - InstanceCategories + - donotoptimize + - DoNotOptimize + type: string + type: object + reliability: + properties: + oom: + properties: + memoryBumpUp: + properties: + applyImmediately: + enum: + - IfAutoDeployEnabled + - ifautodeployenabled + - Always + - always + - Never + - never + type: string + max: + type: string + min: + type: string + percent: + type: string + period: + type: string + type: object + type: object + type: object + schedule: + description: |- + Schedule dictates the frequency of the recommendations. + This can be done using one of the following syntaxes: + - Cron; 10 min + A schedule for every 10 minutes would look like: `schedule: */10 * * * *`. + - ISO-8601 Duration ( https://en.wikipedia.org/wiki/ISO_8601#Durations ) + A schedule for every 1 hour would look like `schedule: PT1H`.\ + this overrides the schedule under Schedules. + we will set this field as optional until we completely deprecate Schedules + type: string + thresholds: + additionalProperties: + properties: + minPercentChange: + minimum: 0 + type: integer + minUnitChange: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + type: object + workloadGroup: + description: |- + WorkloadGroup aggregates usage metrics from related workloads so this + workload's recommendation reflects the group's combined resource usage. + properties: + cluster: + description: Cluster overrides the cluster to include. Defaults + to this workload's cluster. + type: string + excludeTarget: + description: |- + ExcludeTarget removes this workload from its own group (mirror mode), so only + the matched workloads' metrics drive recommendations. Defaults to false. + type: boolean + expression: + description: |- + Expression is a CEL expression matching workloads to include. The `target` + variable refers to this workload and `candidate` to the workload being matched. + type: string + name: + description: Name overrides the workload name to include. Defaults + to this workload's name. + type: string + namespace: + description: Namespace overrides the namespace to include. Defaults + to this workload's namespace. + type: string + resource: + description: Resource overrides the resource type to include. + Defaults to this workload's resource type. + type: string + selector: + description: Selector is a Kubernetes label selector matching + workloads to include (e.g. "app=nginx"). + type: string + type: object + workloadSettings: + additionalProperties: + description: WorkloadSettings provides a way to configure settings + for the entire workload. + properties: + optimizationGoal: + description: Goal is used to configure the aggressiveness of + the recommendations. + enum: + - savings + - Savings + - balanced + - Balanced + - reliability + - Reliability + type: string + type: object + description: Workload allows configuring workload settings. + type: object + workloadTargetRef: + description: TargetRef specifies the kind and name of the workload + object. + properties: + apiVersion: + description: API version of the referent + type: string + kind: + description: |- + Kind of the referent; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + Restricted to [deployments|statefulsets|daemonsets|rollouts] for initial release + enum: + - deployments + - statefulsets + - daemonsets + - rollouts + type: string + name: + description: 'Name of the referent; More info: http://kubernetes.io/docs/user-guide/identifiers#names' + type: string + required: + - kind + - name + type: object + type: object + status: + description: WorkloadOptimizerStatus defines the observed state of Optimizer. + properties: + state: + description: |- + WorkloadOptimizerState communicates the current state of the WorkloadOptimizer. + **NotSynchronized** configuration has not been sent to StormForge. + **Synchronized** configuration has been sent to StormForge. + **Ignored** configuration will not be sent to StormForge. + enum: + - NotSynchronized + - Synchronized + - Ignored + type: string + type: object + type: object + served: true + storage: true + subresources: + status: {} diff --git a/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/NOTES.txt b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/NOTES.txt new file mode 100644 index 00000000..5a208b19 --- /dev/null +++ b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/NOTES.txt @@ -0,0 +1,35 @@ +--- + █████████████████████████ ████████████████████ +█████████████████████████████████████████████████ ██████████████████████ + ██████████████████████████████████████████████ ████████████████████████ + ██████████████████████████████████████████ ████████████████████ + ██████████████████████████████████████ █████████████ + █████████████████████████████████████████ █████████████ + █████████████████████████████████ █████████████ + ██████████████████ ████████████████ + ███████████████ ███████████████████ + ██████████████████████████████████ + ███████████████████████████████████ + ██████████████████████████████████████ + █████████████████████████████████████████████ + █████████████████████████████████████████████████████ + ███████████████████████ ██████████████████████ + █████████████████████ █████████████████████ + +StormForge Optimize Live has been installed! + +When your recommendations are ready, you'll be able to see them at: + + https://app.stormforge.io/ + +Having installation issues? View our Troubleshooting Guide at: + + https://docs.stormforge.io/docs/reference/troubleshooting/ + +--- +{{- if .Values.patchTargetTypes }} + +DEPRECATION WARNING: `patchTargetTypes` is deprecated and will be removed in a +future release. Please use `workloadResourceTypes` and `ownerResourceTypes` to +configure patchTargetTypeDefaults instead. +{{- end }} diff --git a/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/_helpers.tpl b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/_helpers.tpl new file mode 100644 index 00000000..650e31ab --- /dev/null +++ b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/_helpers.tpl @@ -0,0 +1,193 @@ +{{/* +Expand the name of the chart. +*/}} +{{- define "stormforge-agent.name" -}} +{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Create a default fully qualified app name. +We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). +If release name contains chart name it will be used as a full name. +*/}} +{{- define "stormforge-agent.fullname" -}} +{{- if .Values.fullnameOverride }} +{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }} +{{- else }} +{{- $name := default .Chart.Name .Values.nameOverride }} +{{- if contains $name .Release.Name }} +{{- .Release.Name | trunc 63 | trimSuffix "-" }} +{{- else }} +{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }} +{{- end }} +{{- end }} +{{- end }} + +{{/* Component Names */}} + +{{- define "stormforge-agent.agentComponent" -}} +agent +{{- end -}} + +{{- define "stormforge-agent.agentName" -}} +{{ .Release.Name }}-workload-controller +{{- end -}} + +{{- define "stormforge-agent.metricsForwarderComponent" -}} +metrics-forwarder +{{- end -}} + +{{- define "stormforge-agent.metricsForwarderName" -}} +{{ .Release.Name }}-{{ include "stormforge-agent.metricsForwarderComponent" . }} +{{- end -}} + +{{- define "stormforge-agent.envSecret" -}} +{{- if .Values.envSecret }}{{ .Values.envSecret }}{{ else }}{{ printf "%s-env" (include "stormforge-agent.fullname" .) }}{{ end }} +{{- end -}} + +{{- define "stormforge-agent.authSecret" -}} +{{- if .Values.authSecret }}{{ .Values.authSecret }}{{ else }}{{ printf "%s-auth" (include "stormforge-agent.fullname" .) }}{{ end }} +{{- end -}} + +{{- define "stormforge-agent.defaultsConfigMap" -}} +cluster-defaults +{{- end -}} + +{{/* Create chart name and version as used by the chart label */}} +{{- define "stormforge-agent.chart" -}} +{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* Validator and deprecation access function for clusterName */}} +{{- define "stormforge-agent.clusterName" -}} + {{- if ne nil (.Values.AsMap | dig "stormforge" "clusterName" nil) -}} + {{- fail "Chart value `stormforge.clusterName` has been deprecated; please use `clusterName` instead" -}} + {{- end -}} + {{- if gt (semver .Chart.Version).Major 2 -}} + {{- fail "Developer note: remove all .Values.stormforge.clusterName references from templates and values.schema.json before incrementing Chart major version" -}} + {{- end -}} + {{- if and .Values.clusterName .Values.clusterNameSeed -}} + {{- fail "Chart values `clusterName` and `clusterNameSeed` are mutually exclusive! You must set at least one of these values to `null`" -}} + {{- end -}} + + {{/* Usually, the clusterName will simply be given. If given, it is validated by values.schema.json. */}} + {{- $name := .Values.clusterName -}} + + {{/* As a mutually exclusive alternative, the name can be generated from a seed. This is often used to de-duplicate clusters + with the same name but on different EKS regions (the region will be given as a clusterNameSuffix). If using a name seed, + we will sanitize and munge the final name value to ensure that the string passed to the API is unique and compliant. */}} + {{- if .Values.clusterNameSeed -}} + {{- $name = .Values.clusterNameSeed | lower | replace "_" "-" -}} + {{/* Ensure that a generated name will have a legal length, even if the seed did not. */}} + {{- $clusterNameMaxLen := 63 -}} + {{- if .Values.clusterNameSuffix -}} + {{/* Account for the length of a suffix that will be added, if configured. */}} + {{- $clusterNameMaxLen = sub 63 (len .Values.clusterNameSuffix | add1) | int -}} + {{- end -}} + {{- if gt (len $name | int) $clusterNameMaxLen -}} + {{- $nameHash := sha1sum (printf "%s-%s" .Values.clusterNameSeed .Values.clusterNameSuffix) | lower | trunc 8 -}} + {{- $name = printf "%s-%s" ($name | trunc (sub $clusterNameMaxLen 9 | int)) $nameHash -}} + {{- end -}} + {{- end -}} + + {{/* Add a suffix to the name, if specified. Fall through to `required` below if there is no name. */}} + {{- if and $name .Values.clusterNameSuffix -}} + {{- $name = printf "%s-%s" $name .Values.clusterNameSuffix -}} + {{- if gt (len $name) 63 -}} + {{- fail (printf "Chart values `clusterName` and `clusterNameSuffix` are invalid together. \"%s\" exceeds 63 characters." $name) -}} + {{- end -}} + {{- end -}} + + {{/* At this point, if the input was valid, we should have a value. */}} + {{- required "Chart value `clusterName` is required!" $name -}} +{{- end -}} + +{{/* Common labels */}} + +{{- define "stormforge-agent.commonSelectorLabels" -}} +app.kubernetes.io/name: {{ include "stormforge-agent.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end }} + +{{- define "stormforge-agent.agentSelectorLabels" -}} +{{- include "stormforge-agent.commonSelectorLabels" . }} +app.kubernetes.io/component: {{ include "stormforge-agent.agentComponent" . }} +{{- end }} + +{{- define "stormforge-agent.metricsForwarderSelectorLabels" -}} +{{- include "stormforge-agent.commonSelectorLabels" . }} +app.kubernetes.io/component: {{ include "stormforge-agent.metricsForwarderComponent" . }} +{{- end }} + +{{- define "stormforge-agent.labels" -}} +helm.sh/chart: {{ include "stormforge-agent.chart" . }} +{{ include "stormforge-agent.commonSelectorLabels" . }} +{{- if .Chart.AppVersion }} +app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} +{{- end }} +{{- with .Values.commonMetaLabels}} +{{ toYaml . }} +{{- end }} +app.kubernetes.io/managed-by: Helm +{{- end }} + +{{/* +Create the name of the service account to use +*/}} +{{- define "stormforge-agent.serviceAccountName" -}} +{{- if .Values.serviceAccount.create }} +{{- default (include "stormforge-agent.fullname" .) .Values.serviceAccount.name }} +{{- else }} +{{- default "default" .Values.serviceAccount.name }} +{{- end }} +{{- end }} + +{{/* +Create the clusterrolebinding view name +*/}} +{{- define "stormforge-agent.roleBindingViewName" -}} +{{- printf "%s-view" (include "stormforge-agent.fullname" .) -}} +{{- end -}} + +{{/* StormForge Labels to keep */}} +{{- define "stormforge-agent.workload.labels-to-keep" -}} +{{- printf "%s" + (list + "__name__" + "cluster_name" + "container" + "instance" + "name" + "node" + "kubernetes_io_arch" + "node_kubernetes_io_instance_type" + "pod" + "topology_kubernetes_io_region" + "kubernetes_io_hostname" + "topology_kubernetes_io_zone" + "kubernetes_io_os" + "namespace" + "address" + "job" + "interface" + | join "|") -}} +{{- end -}} + +{{/* StormForge Labels to keep after ingest */}} +{{- define "stormforge-agent.workload.labels-to-keep-postingest" -}} +{{- $labels := include "stormforge-agent.workload.labels-to-keep" . -}} +{{- printf "^(%s)$" ($labels) -}} +{{- end -}} + +{{/* StormForge Labels to keep during ingest, same as before plus all metadata because they are required for autogenerated metrics */}} +{{- define "stormforge-agent.workload.labels-to-keep-duringingest" -}} +{{- $postlabels := include "stormforge-agent.workload.labels-to-keep" . -}} +{{- $labelsspace := cat $postlabels "|__.*" -}} +{{- $labels := $labelsspace | replace " " "" -}} +{{- printf "^(%s)$" ($labels) -}} +{{- end -}} + +{{/* Next major Chart version should rename component Values fields */}} +{{- if gt (semver .Chart.Version).Major 2 -}} + {{- fail "Developer note: rename .Values.workload to .Values.workloadController and .Values.prom to .Values.metricsForwarder before incrementing Chart major version" -}} +{{- end -}} \ No newline at end of file diff --git a/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/auth-secret.yaml b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/auth-secret.yaml new file mode 100644 index 00000000..64282c94 --- /dev/null +++ b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/auth-secret.yaml @@ -0,0 +1,23 @@ +{{- if .Values.manageAuthSecret -}} +--- +apiVersion: v1 +kind: Secret +type: Opaque +metadata: + name: {{ include "stormforge-agent.authSecret" . | quote }} + namespace: {{ .Release.Namespace | quote }} + labels: + {{- include "stormforge-agent.labels" . | nindent 4 }} +data: + {{- if .Values.standbyMode }} + STORMFORGE_CLIENT_ID: {{ "standbyMode" | b64enc | quote }} + STORMFORGE_CLIENT_SECRET: {{ "standbyMode" | b64enc | quote }} + STORMFORGE_ISSUER: {{ required "Chart value `stormforge.address` is required!" .Values.stormforge.address | b64enc | quote }} + STORMFORGE_SERVER: {{ required "Chart value `stormforge.address` is required!" .Values.stormforge.address | b64enc | quote }} + {{- else }} + STORMFORGE_CLIENT_ID: {{ required "Chart value `authorization.clientID` is required!" .Values.authorization.clientID | b64enc | quote }} + STORMFORGE_CLIENT_SECRET: {{ required "Chart value `authorization.clientSecret` is required!" .Values.authorization.clientSecret | b64enc | quote }} + STORMFORGE_ISSUER: {{ required "Chart value `authorization.issuer` is required!" .Values.authorization.issuer | b64enc | quote }} + STORMFORGE_SERVER: {{ required "Chart value `stormforge.address` is required!" .Values.stormforge.address | b64enc | quote }} + {{- end -}} +{{- end -}} \ No newline at end of file diff --git a/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/cluster-defaults-configmap.yaml b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/cluster-defaults-configmap.yaml new file mode 100644 index 00000000..352364ad --- /dev/null +++ b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/cluster-defaults-configmap.yaml @@ -0,0 +1,63 @@ +{{- if .Values.clusterDefaultConfig }} +{{- $defaults := dict }} +{{- $annotationKeys := dict + "schedule" "live.stormforge.io/schedule" + "learningPeriod" "live.stormforge.io/learning-period" + "metricsFrom" "live.stormforge.io/metrics-from" + "applyMethod" "live.stormforge.io/apply.method" + "applyMaxPercentIncrease" "live.stormforge.io/apply.max-percent-increase" + "applyMaxPercentDecrease" "live.stormforge.io/apply.max-percent-decrease" + "autoDeploy" "live.stormforge.io/auto-deploy" + "autoDeployThresholdsCpuUnit" "live.stormforge.io/auto-deploy.thresholds.cpu.unit" + "autoDeployThresholdsCpuPercent" "live.stormforge.io/auto-deploy.thresholds.cpu.percent" + "autoDeployThresholdsMemoryUnit" "live.stormforge.io/auto-deploy.thresholds.memory.unit" + "autoDeployThresholdsMemoryPercent" "live.stormforge.io/auto-deploy.thresholds.memory.percent" + "reliabilityOomMemoryBumpUpPeriod" "live.stormforge.io/reliability.oom.memory-bump-up.period" + "reliabilityOomMemoryBumpUpPercent" "live.stormforge.io/reliability.oom.memory-bump-up.percent" + "reliabilityOomMemoryBumpUpMin" "live.stormforge.io/reliability.oom.memory-bump-up.min" + "reliabilityOomMemoryBumpUpMax" "live.stormforge.io/reliability.oom.memory-bump-up.max" + "reliabilityOomMemoryBumpUpApplyImmediately" "live.stormforge.io/reliability.oom.memory-bump-up.apply-immediately" + "cpuOptimizationGoal" "live.stormforge.io/cpu.optimization-goal" + "memoryOptimizationGoal" "live.stormforge.io/memory.optimization-goal" + "hpaCpuTargetUtilizationMin" "live.stormforge.io/hpa.cpu.target-utilization.min" + "hpaCpuTargetUtilizationMax" "live.stormforge.io/hpa.cpu.target-utilization.max" + "hpaCpuTargetUtilizationPatchPath" "live.stormforge.io/hpa.cpu.target-utilization.patch-path" + "hpaMemoryTargetUtilizationMin" "live.stormforge.io/hpa.memory.target-utilization.min" + "hpaMemoryTargetUtilizationMax" "live.stormforge.io/hpa.memory.target-utilization.max" + "hpaMemoryTargetUtilizationPatchPath" "live.stormforge.io/hpa.memory.target-utilization.patch-path" + "containersCpuOptimizationPolicy" "live.stormforge.io/containers.cpu.optimization-policy" + "containersCpuRequestsMin" "live.stormforge.io/containers.cpu.requests.min" + "containersCpuRequestsMax" "live.stormforge.io/containers.cpu.requests.max" + "containersCpuRequestsPatchPath" "live.stormforge.io/containers.cpu.requests.patch-path" + "containersCpuLimitsMin" "live.stormforge.io/containers.cpu.limits.min" + "containersCpuLimitsMax" "live.stormforge.io/containers.cpu.limits.max" + "containersCpuLimitsPatchPath" "live.stormforge.io/containers.cpu.limits.patch-path" + "containersCpuLimitsLimitRequestRatio" "live.stormforge.io/containers.cpu.limits.limit-request-ratio" + "containersMemoryOptimizationPolicy" "live.stormforge.io/containers.memory.optimization-policy" + "containersMemoryRequestsMin" "live.stormforge.io/containers.memory.requests.min" + "containersMemoryRequestsMax" "live.stormforge.io/containers.memory.requests.max" + "containersMemoryRequestsPatchPath" "live.stormforge.io/containers.memory.requests.patch-path" + "containersMemoryLimitsMin" "live.stormforge.io/containers.memory.limits.min" + "containersMemoryLimitsMax" "live.stormforge.io/containers.memory.limits.max" + "containersMemoryLimitsLimitRequestRatio" "live.stormforge.io/containers.memory.limits.limit-request-ratio" + "containersMemoryLimitsPatchPath" "live.stormforge.io/containers.memory.limits.patch-path" + "containersJavaMaxHeapMin" "live.stormforge.io/containers.java.max-heap.min" + "containersJavaMaxHeapMax" "live.stormforge.io/containers.java.max-heap.max" + "containersJavaMaxHeapPatchPath" "live.stormforge.io/containers.java.max-heap.patch-path" + "containersJavaMaxHeapPatchFormat" "live.stormforge.io/containers.java.max-heap.patch-format" +-}} +{{- range $k, $v := .Values.clusterDefaultConfig -}} + {{- $_ := set $defaults (get $annotationKeys $k) ($v | toString) -}} +{{- end -}} +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ include "stormforge-agent.defaultsConfigMap" . }} + labels: + {{- include "stormforge-agent.labels" . | nindent 4 }} + namespace: {{ .Release.Namespace }} +data: + cluster-defaults.yaml: | + {{- $defaults | toYaml | nindent 4 }} +{{- end }} diff --git a/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/clusterrole-crd.yaml b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/clusterrole-crd.yaml new file mode 100644 index 00000000..cc39599e --- /dev/null +++ b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/clusterrole-crd.yaml @@ -0,0 +1,29 @@ +{{- if .Values.rbac.create -}} +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: {{ printf "%s-crd" (include "stormforge-agent.fullname" .) }} + labels: + rbac.authorization.k8s.io/aggregate-to-view: "true" + {{- include "stormforge-agent.labels" . | nindent 4 }} +rules: +- apiGroups: + - optimize.stormforge.io + resources: + - optimizationconfigurations + - clusteroptimizationconfigurations + - workloadoptimizers + verbs: + - get + - list + - watch +- apiGroups: + - optimize.stormforge.io + resources: + - workloadoptimizers/status + verbs: + - get + - list + - watch +{{- end -}} \ No newline at end of file diff --git a/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/clusterrole.yaml b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/clusterrole.yaml new file mode 100644 index 00000000..19ec9a84 --- /dev/null +++ b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/clusterrole.yaml @@ -0,0 +1,172 @@ +{{- if .Values.rbac.create -}} +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: {{ include "stormforge-agent.fullname" . }} + labels: {{- include "stormforge-agent.labels" . | nindent 4 }} +rules: +- apiGroups: + - "" + resources: + - nodes + - nodes/proxy + verbs: + - get + - list + - watch +- apiGroups: + - optimize.stormforge.io + resources: + - workloadoptimizers/status + verbs: + - create + - delete + - patch + - update +- nonResourceURLs: ["/metrics"] + verbs: ["get"] + +{{- if .Values.openshift }}{{ "\n" }} +# Rule for: OpenShift +- apiGroups: + - security.openshift.io + resources: + - securitycontextconstraints + verbs: + - use + resourceNames: + - anyuid +# Rule for: OpenShift operator-owned workloads +- apiGroups: + - config.openshift.io + resources: + - clusterversions + verbs: + - get + - list + - watch +- apiGroups: + - operator.openshift.io + resources: + - dnses + - ingresscontrollers + verbs: + - get + - list + - watch +{{- end }} + +{{- $rbacTypes := + concat (.Values.workloadResourceTypes | default list) + (.Values.ownerResourceTypes | default list) + (.Values.patchTargetTypes | default list) -}} +{{- with $rbacTypes }}{{ "\n" }} +{{- $gr := list -}} + {{- range . -}} + {{- $gr = append $gr (dict "group" .group "resource" .resource) -}} +{{- end -}} +# From Helm parameters: `workloadResourceTypes`, `ownerResourceTypes`, `patchTargetTypes` +{{- range ($gr | uniq) }} +- apiGroups: + - {{ .group | quote }} + resources: + - {{ .resource | quote }} + verbs: + - get + - list + - watch +{{- end }} +{{- end }} + +{{- with .Values.featureGates }} +{{- if .optimizeSparkOperator }}{{ "\n" }} +# Rule for feature: Optimize Spark Operator +- apiGroups: + - sparkoperator.k8s.io + resources: + - scheduledsparkapplications + - sparkapplications + verbs: + - get + - list + - watch +{{- end }} +{{- if .optimizeGithubARC }}{{ "\n" }} +# Rule for feature: Optimize Github ARC +- apiGroups: + - actions.github.com + resources: + - autoscalinglisteners + - autoscalingrunnersets + - ephemeralrunners + - ephemeralrunnersets + verbs: + - get + - list + - watch +{{- end }} +{{- end }} + +{{- with .Values.rbac -}} +{{- if not .useViewRole }}{{ "\n" }} +# Minimum RBAC for Built-in Resources +- apiGroups: + - optimize.stormforge.io + resources: + - optimizationconfigurations + - clusteroptimizationconfigurations + - workloadoptimizers + verbs: + - get + - list + - watch +- apiGroups: + - "" + resources: + - namespaces + - pods + - replicationcontrollers + - replicationcontrollers/scale + verbs: + - get + - list + - watch +- apiGroups: + - apps + resources: + - replicasets + - replicasets/scale + - statefulsets + - statefulsets/scale + - daemonsets + - deployments + - deployments/scale + verbs: + - get + - list + - watch +- apiGroups: + - autoscaling + resources: + - horizontalpodautoscalers + verbs: + - get + - list + - watch +- apiGroups: + - monitoring.coreos.com + resources: + - prometheuses + verbs: + - get + - list + - watch +{{- end}} +{{- if .additional }}{{ "\n" }} +# From Helm parameter: `rbac.additional` +{{ .additional }} +{{- end }} +{{- end }} + +{{- end -}} diff --git a/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/clusterrolebinding.yaml b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/clusterrolebinding.yaml new file mode 100644 index 00000000..10891f59 --- /dev/null +++ b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/clusterrolebinding.yaml @@ -0,0 +1,51 @@ +{{- if .Values.rbac.create -}} +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: {{ include "stormforge-agent.fullname" . }} + labels: + {{- include "stormforge-agent.labels" . | nindent 4 }} +subjects: +- kind: ServiceAccount + name: {{ include "stormforge-agent.serviceAccountName" . }} + namespace: {{ .Release.Namespace }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: {{ include "stormforge-agent.fullname" . }} + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: {{ printf "%s-crd" (include "stormforge-agent.fullname" .) }} + labels: + {{- include "stormforge-agent.labels" . | nindent 4 }} +subjects: +- kind: ServiceAccount + name: {{ include "stormforge-agent.serviceAccountName" . }} + namespace: {{ .Release.Namespace }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: {{ printf "%s-crd" (include "stormforge-agent.fullname" .) }} + +{{ if .Values.rbac.useViewRole -}} +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: {{ include "stormforge-agent.roleBindingViewName" . }} + labels: + {{- include "stormforge-agent.labels" . | nindent 4 }} +subjects: +- kind: ServiceAccount + name: {{ include "stormforge-agent.serviceAccountName" . }} + namespace: {{ .Release.Namespace }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: view +{{- end }} +{{- end }} diff --git a/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/env-secret.yaml b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/env-secret.yaml new file mode 100644 index 00000000..6413234d --- /dev/null +++ b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/env-secret.yaml @@ -0,0 +1,26 @@ +{{- $values := dict + "STORMFORGE_CLUSTER_NAME" (required "Chart value `clusterName` is required!" (include "stormforge-agent.clusterName" .)) +-}} + +{{- if .Values.proxyUrl -}} + {{- $_ := set $values "HTTP_PROXY" .Values.proxyUrl -}} + {{- $_ := set $values "HTTPS_PROXY" .Values.proxyUrl -}} +{{- end -}} +{{- if .Values.noProxy -}} + {{- $_ := set $values "NO_PROXY" .Values.noProxy -}} +{{- end -}} + +{{- range .Values.extraEnvVars -}}{{- $_ := set $values .name .value -}}{{- end -}} + +{{- range $k, $v := $values -}}{{- $_ := set $values $k ($v | b64enc) -}}{{- end -}} +--- +apiVersion: v1 +kind: Secret +type: Opaque +metadata: + name: {{ include "stormforge-agent.envSecret" . | quote }} + namespace: {{ .Release.Namespace | quote }} + labels: + {{- include "stormforge-agent.labels" . | nindent 4 }} +data: + {{- $values | toYaml | nindent 2 }} diff --git a/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/metrics-forwarder/configmap.yaml b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/metrics-forwarder/configmap.yaml new file mode 100644 index 00000000..b21506fe --- /dev/null +++ b/packs/stormforge-agent-2.28.5/charts/stormforge-agent/templates/metrics-forwarder/configmap.yaml @@ -0,0 +1,393 @@ +{{- if and .Values.workload.denyNamespaces .Values.workload.allowNamespaces -}} +{{- fail "Chart values `workload.denyNamespaces` and `workload.allowNamespaces` are mutually exclusive! You must set at least one of these values to `null`" -}} +{{- end -}} +{{- if and .Values.enableCostMetrics (or .Values.workload.denyNamespaces .Values.workload.allowNamespaces) -}} +{{- fail "Chart value `enableCostMetrics` is mutually exclusive with `workload.allowNamespaces` and `workload.denyNamespaces`! You must set both of these values to `null` to use `enableCostMetrics`" -}} +{{- end -}} + +{{/* + Render relabel_configs shared between two of our job definitions. + + Usage: + {{- include "stormforge.commonNodeRelabelConfigs" + (dict "metricsPath" "/metrics/cadvisor" "context" $) -}} +*/}} +{{- define "stormforge.commonNodeRelabelConfigs" -}} +- action: labelmap + regex: __meta_kubernetes_node_label_(.+) +{{- if .context.Values.gkeAutopilot }} +# Use target configuration specific to GKE Autopilot +- target_label: __scheme__ + replacement: "http" +- target_label: __address__ + source_labels: [__address__] + regex: "(.+):(.+)" + replacement: "${1}:10255" +- target_label: __metrics_path__ + replacement: {{ .metricsPath | quote }} +{{- else }} +# Use kubeproxy to scrape node data +- target_label: __address__ + replacement: kubernetes.default.svc:443 +- source_labels: [__meta_kubernetes_node_name] + regex: (.+) + target_label: __metrics_path__ + replacement: {{ printf "/api/v1/nodes/${1}/proxy%s" .metricsPath | quote }} +{{- end }} +# Inject a "node" label to match StormForge agent metrics on +- source_labels: [__meta_kubernetes_node_name] + target_label: node +- regex: {{ include "stormforge-agent.workload.labels-to-keep-duringingest" . }} + action: labelkeep +{{- end -}} + +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ include "stormforge-agent.metricsForwarderName" . }} + namespace: {{ .Release.Namespace }} + {{- if .Values.prom.configMapAnnotations }} + annotations: {{ toYaml .Values.prom.configMapAnnotations | nindent 4 }} + {{- end }} + labels: + app.kubernetes.io/component: {{ include "stormforge-agent.metricsForwarderComponent" . }} + {{- include "stormforge-agent.labels" . | nindent 4 }} +data: + prometheus.yml: | + {{- if .Values.standbyMode }} + --- + # Helm installed in standby mode. + {{- else }} + --- + global: + scrape_interval: {{ .Values.prom.scrapeInterval | quote }} + scrape_timeout: {{ .Values.prom.scrapeTimeout | quote }} + external_labels: + cluster_name: "$STORMFORGE_CLUSTER_NAME" + + remote_write: + - url: {{ .Values.prom.remoteWriteUrl }} + name: application + {{- if .Values.prom.remoteTimeout }} + remote_timeout: {{ .Values.prom.remoteTimeout }} + {{- end }} + proxy_from_environment: true + oauth2: + proxy_from_environment: true + client_id: "$STORMFORGE_CLIENT_ID" + client_secret_file: /prometheus/client/client_secret_file + token_url: {{ .Values.authorization.issuer }}oauth/token + endpoint_params: + audience: {{ .Values.stormforge.address }} + cluster: "$STORMFORGE_CLUSTER_NAME" + queue_config: + {{- toYaml .Values.prom.remoteWriteQueueConfig | nindent 10 }} + write_relabel_configs: + - target_label: cluster_name + replacement: "$STORMFORGE_CLUSTER_NAME" + + {{- if .Values.nodeCPUPressureDetection }} + - url: http://{{ include "stormforge-agent.agentName" . }}:8080/write + name: node-cpu-pressure-detection + write_relabel_configs: + - source_labels: [__name__] + regex: ^container_pressure_cpu_(waiting|stalled)_seconds_total$ + action: keep + # The cadvisor job dropped all but the root-cgroup PSI metrics (id="/"), + # then stripped out the id label. Restore id="/" for this write. + - target_label: id + replacement: / + - source_labels: [__name__] + regex: ^container_pressure_cpu_waiting_seconds_total$ + target_label: __name__ + replacement: sf_container_cpu_pressure_waiting_seconds_total + - source_labels: [__name__] + regex: ^container_pressure_cpu_stalled_seconds_total$ + target_label: __name__ + replacement: sf_container_cpu_pressure_stalled_seconds_total + {{- end }} + + scrape_configs: + - job_name: 'kubernetes-cadvisor' + scheme: https + tls_config: + ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt + bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token + kubernetes_sd_configs: + - role: node + honor_timestamps: {{ .Values.prom.honorTimestamps }} + relabel_configs: {{- include "stormforge.commonNodeRelabelConfigs" + (dict "metricsPath" "/metrics/cadvisor" "context" $) | nindent 8 }} + metric_relabel_configs: + # Limit input to only these metrics + - regex: ^{{- list + "container_cpu_usage_seconds_total" + "container_memory_working_set_bytes" + "container_cpu_cfs_throttled_seconds_total" + "container_cpu_cfs_throttled_periods_total" + "container_cpu_cfs_periods_total" + "container_memory_max_usage_bytes" + | concat (.Values.enableCostMetrics | ternary + (list "container_network_transmit_bytes_total") + (list)) + | concat (.Values.nodeCPUPressureDetection | ternary + (list "container_pressure_cpu_waiting_seconds_total" + "container_pressure_cpu_stalled_seconds_total") + (list)) + | concat (.Values.prom.extraCAdvisorMetrics | default list) + | uniq + | join "|" -}}$ + source_labels: [ __name__ ] + action: keep + # Earmark network metrics to keep even without a container label + - source_labels: [__name__] + regex: container_network_transmit_bytes_total + action: replace + target_label: __keep + replacement: t + # The /metrics/resource node_* metrics are just cAdvisor root-cgroup id="/" + # samples. Derive them here instead of a separate scrape job. Keep-earmark + # the root series, and save with the /metrics/resource equivalent names. + - source_labels: [__name__, id] + regex: container_(cpu_usage_seconds_total|memory_working_set_bytes);/ + action: replace + target_label: __keep + replacement: t + - source_labels: [__name__, id] + regex: container_cpu_usage_seconds_total;/ + action: replace + target_label: __name__ + replacement: node_cpu_usage_seconds_total + - source_labels: [__name__, id] + regex: container_memory_working_set_bytes;/ + action: replace + target_label: __name__ + replacement: node_memory_working_set_bytes + {{- if .Values.nodeCPUPressureDetection }} + # Drop all PSI series except for root-cgroup id="/". RE2 has no negative + # lookahead, so use an alternation regex. + - source_labels: [__name__, id] + regex: container_pressure_cpu_(waiting|stalled)_seconds_total;(|/.+|[^/].*) + action: drop + # Root-cgroup PSI series carry none of the pod/image/name/namespace labels, + # so earmark them to survive the drops below. + - source_labels: [__name__, id] + regex: container_pressure_cpu_(waiting|stalled)_seconds_total;/ + action: replace + target_label: __keep + replacement: t + {{- end }} + # Remove non-earmarked metrics that don't have a 'container' label + - source_labels: [ container, __keep ] + regex: ^(POD|);$ + action: drop + # The empty-label drops spare __keep-earmarked series (root-cgroup PSI + # carries none of these labels); for everything else the joined + # "