-
Notifications
You must be signed in to change notification settings - Fork 32
feat: Write roles fingerprints to /var/log/sysroles.jsonl [citest_skip] #514
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -7,29 +7,148 @@ | |
| DOCUMENTATION = """ | ||
| --- | ||
| module: sr_fingerprint | ||
| short_description: Write a message string to syslog using Ansible C(module.log) function. | ||
| short_description: Write role fingerprint data to syslog and optionally to a JSONL log file | ||
| description: | ||
| - Writes the given string to the system log using Ansible C(module.log) function. | ||
| - Collects role fingerprint data into a canonical record and writes it to | ||
| syslog using Ansible C(module.log) as C(key=value) pairs. | ||
| - Optionally appends the same record as a JSON line to a log file | ||
| (one JSON object per line, JSONL format), by default | ||
| C(/var/log/sysroles.jsonl). | ||
| - Playbook variables are not available inside modules automatically. Roles | ||
| pass C(role_name), C(role_path), C(ansible_play_hosts_all), | ||
| C(distribution), and C(distribution_version) from the task. | ||
| - C(ansible_check_mode) is collected from the module execution context. | ||
| - Intended for role-internal or diagnostic use. | ||
| author: Rich Megginson (@richm) | ||
| options: | ||
| sr_message: | ||
| description: Text to record in syslog. | ||
| status: | ||
| description: Role execution status. | ||
| type: str | ||
| required: true | ||
| choices: | ||
| - begin | ||
| - success | ||
| write_log_file: | ||
| description: >- | ||
| If C(true), append fingerprint data to the JSONL log file. | ||
| Defaults to C(false). | ||
| type: bool | ||
| default: false | ||
| log_file: | ||
| description: >- | ||
| Path to the JSONL log file. A lock sidecar (C(<log_file>.lock)) | ||
| is created next to the log file for cross-process safety. | ||
| type: path | ||
| default: /var/log/sysroles.jsonl | ||
| max_log_size: | ||
| description: >- | ||
| Maximum log file size in bytes. When appending a new record | ||
| would exceed this limit, the oldest records are removed first. | ||
| Set to C(0) to disable trimming. | ||
| type: int | ||
| default: 2000000 | ||
| role_name: | ||
| description: Name of the role, typically C({{ role_name }}). | ||
| type: str | ||
| required: true | ||
| role_path: | ||
| description: Path to the role, typically C({{ role_path }}). | ||
| type: path | ||
| required: true | ||
| ansible_play_hosts_all: | ||
| description: >- | ||
| All hosts in the play, typically C({{ ansible_play_hosts_all }}). | ||
| Used to derive C(play_hosts_number). | ||
| type: list | ||
| elements: str | ||
| required: true | ||
| distribution: | ||
| description: >- | ||
| OS distribution name, typically | ||
| C({{ ansible_facts["distribution"] }}). | ||
| type: str | ||
| default: "" | ||
| distribution_version: | ||
| description: >- | ||
| OS distribution version, typically | ||
| C({{ ansible_facts["distribution_version"] }}). | ||
| type: str | ||
| default: "" | ||
| """ | ||
|
|
||
| EXAMPLES = """ | ||
| - name: Record a fingerprint message in syslog | ||
| - name: Record role begin fingerprint to syslog only (not log file) | ||
| sr_fingerprint: | ||
| status: begin | ||
| role_name: bootloader | ||
| role_path: "{{ role_path }}" | ||
| ansible_play_hosts_all: "{{ ansible_play_hosts_all }}" | ||
| distribution: "{{ ansible_facts['distribution'] }}" | ||
| distribution_version: "{{ ansible_facts['distribution_version'] }}" | ||
| write_log_file: false | ||
|
|
||
| - name: Record role success fingerprint | ||
| sr_fingerprint: | ||
| sr_message: "system_role:ROLENAME" | ||
| status: success | ||
| role_name: bootloader | ||
| role_path: "{{ role_path }}" | ||
| ansible_play_hosts_all: "{{ ansible_play_hosts_all }}" | ||
| distribution: "{{ ansible_facts['distribution'] }}" | ||
| distribution_version: "{{ ansible_facts['distribution_version'] }}" | ||
| write_log_file: true | ||
| """ | ||
|
|
||
| RETURN = r""" # """ | ||
| RETURN = r""" | ||
| fingerprint: | ||
| description: The fingerprint record written to syslog and optionally to the log file. | ||
| returned: always | ||
| type: dict | ||
| sample: | ||
| date: "2026-08-03T10:15:00+02:00" | ||
| role_name: network | ||
| role_path: /usr/share/ansible/roles/linux-system-roles.network | ||
| status: success | ||
| ansible_version: "2.16.3" | ||
| managed_node_distro: RedHat-9.4 | ||
| play_hosts_number: 3 | ||
| ansible_check_mode: false | ||
| message: | ||
| description: Informational message shown in check mode. | ||
| returned: check mode | ||
| type: str | ||
| sample: "Check mode: message not logged - [date=... role_name=...]" | ||
| jsonl_row: | ||
| description: The JSON line that would be appended to the log file. | ||
| returned: check mode and O(write_log_file=true) | ||
| type: str | ||
| log_file: | ||
| description: Path to the log file that would be written. | ||
| returned: check mode and O(write_log_file=true) | ||
| type: str | ||
| """ | ||
|
|
||
| from ansible.module_utils.basic import AnsibleModule | ||
|
|
||
| import datetime | ||
| import errno | ||
| import fcntl | ||
| import json | ||
| import os | ||
| import stat | ||
| import tempfile | ||
|
|
||
| FINGERPRINT_FIELDS = ( | ||
| "date", | ||
| "role_name", | ||
| "role_path", | ||
| "status", | ||
| "ansible_version", | ||
| "managed_node_distro", | ||
| "play_hosts_number", | ||
| "ansible_check_mode", | ||
| ) | ||
|
|
||
| FINGERPRINT_SYSLOG_SEPARATOR = " " | ||
|
|
||
|
|
||
| def _local_iso8601_no_microseconds(): | ||
|
|
@@ -51,33 +170,192 @@ def _local_iso8601_no_microseconds(): | |
| return datetime.datetime.now(utc).astimezone().replace(microsecond=0).isoformat() | ||
|
|
||
|
|
||
| def run_module(): | ||
| module_args = dict( | ||
| sr_message=dict(type="str", required=True), | ||
| ) | ||
| def _ensure_parent_dir(path): | ||
| parent = os.path.dirname(path) | ||
| if not parent: | ||
| return | ||
| if os.path.isdir(parent): | ||
| return | ||
| try: | ||
| os.makedirs(parent) | ||
| except OSError as exc: | ||
| # another process may have created the directory | ||
| if exc.errno != errno.EEXIST or not os.path.isdir(parent): | ||
| raise | ||
|
|
||
| module = AnsibleModule( | ||
| argument_spec=module_args, | ||
| supports_check_mode=True, | ||
| ) | ||
|
|
||
| log_message = "%s %s" % ( | ||
| module.params["sr_message"], | ||
| _local_iso8601_no_microseconds(), | ||
| ) | ||
| def _format_fingerprint_jsonl(record): | ||
| """Format the canonical fingerprint record as a single JSON line.""" | ||
| return json.dumps(record, separators=(",", ":"), sort_keys=False) | ||
|
|
||
|
|
||
| def _trim_log_file(log_file, size_needed): | ||
| """Remove oldest records until the file can accommodate size_needed bytes.""" | ||
| with open(log_file, "r") as log_fd: | ||
| lines = log_fd.readlines() | ||
| size_removed = 0 | ||
| while lines and size_removed < size_needed: | ||
| size_removed += len(lines.pop(0)) | ||
| orig_stat = os.stat(log_file) | ||
| dir_name = os.path.dirname(log_file) or "." | ||
| fd, tmp_path = tempfile.mkstemp(dir=dir_name, suffix=".tmp") | ||
| try: | ||
| os.fchmod(fd, stat.S_IMODE(orig_stat.st_mode)) | ||
| try: | ||
| os.fchown(fd, orig_stat.st_uid, orig_stat.st_gid) | ||
| except OSError: | ||
| # not running as root; keep default ownership | ||
| pass | ||
| with os.fdopen(fd, "w") as tmp_fd: | ||
| tmp_fd.writelines(lines) | ||
| tmp_fd.flush() | ||
| os.fsync(tmp_fd.fileno()) | ||
| os.rename(tmp_path, log_file) | ||
| except BaseException: | ||
| try: | ||
| os.unlink(tmp_path) | ||
| except OSError: | ||
| # already removed or never created | ||
| pass | ||
| raise | ||
|
|
||
|
|
||
| def _write_jsonl_log(log_file, record, max_size=0): | ||
| _ensure_parent_dir(log_file) | ||
| new_line = _format_fingerprint_jsonl(record) + "\n" | ||
| lock_path = log_file + ".lock" | ||
| lock_fd = open(lock_path, "w") | ||
| try: | ||
| fcntl.flock(lock_fd, fcntl.LOCK_EX) | ||
| try: | ||
| cur_size = os.path.getsize(log_file) | ||
| except OSError: | ||
| # file does not exist yet | ||
| cur_size = 0 | ||
| if max_size > 0 and cur_size + len(new_line) > max_size and cur_size > 0: | ||
| _trim_log_file(log_file, len(new_line)) | ||
| with open(log_file, "a") as log_fd: | ||
| log_fd.write(new_line) | ||
| finally: | ||
| fcntl.flock(lock_fd, fcntl.LOCK_UN) | ||
| lock_fd.close() | ||
|
|
||
|
|
||
| def _get_managed_node_distro(distribution, distribution_version): | ||
| if distribution and distribution_version: | ||
| return "%s-%s" % (distribution, distribution_version) | ||
| return "unknown" | ||
|
|
||
|
|
||
| def _get_play_hosts_number(play_hosts_all): | ||
| return len(play_hosts_all) | ||
|
|
||
|
|
||
| def _get_ansible_version(module): | ||
| version = getattr(module, "ansible_version", None) | ||
| if version: | ||
| return version | ||
| return "unknown" | ||
|
|
||
|
|
||
| def _get_check_mode(module): | ||
| return bool(getattr(module, "check_mode", False)) | ||
|
|
||
|
|
||
| def _collect_fingerprint_record(module, status): | ||
| """Build the canonical fingerprint record used by all output formatters.""" | ||
| return { | ||
| "date": _local_iso8601_no_microseconds(), | ||
| "role_name": module.params["role_name"], | ||
| "role_path": module.params["role_path"], | ||
| "status": status, | ||
| "ansible_version": _get_ansible_version(module), | ||
| "managed_node_distro": _get_managed_node_distro( | ||
| module.params["distribution"], module.params["distribution_version"] | ||
| ), | ||
| "play_hosts_number": _get_play_hosts_number( | ||
| module.params["ansible_play_hosts_all"] | ||
| ), | ||
| "ansible_check_mode": _get_check_mode(module), | ||
| } | ||
|
|
||
|
|
||
| def _fingerprint_record_items(record): | ||
| return [(field, record[field]) for field in FINGERPRINT_FIELDS] | ||
|
|
||
|
|
||
| def _format_fingerprint_key_value(field, value): | ||
| text = "" if value is None else str(value) | ||
| if any(char in text for char in ' "='): | ||
| return '%s="%s"' % (field, text.replace('"', '""')) | ||
| return "%s=%s" % (field, text) | ||
|
Comment on lines
+287
to
+291
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔒 Security & Privacy | 🟠 Major | ⚡ Quick win Escape CR and LF characters before writing syslog fields. A value containing 🤖 Prompt for AI Agents |
||
|
|
||
|
|
||
| def _format_fingerprint_syslog(record): | ||
| """Format the canonical fingerprint record as key=value syslog text.""" | ||
| pairs = [ | ||
| _format_fingerprint_key_value(field, value) | ||
| for field, value in _fingerprint_record_items(record) | ||
| ] | ||
| return FINGERPRINT_SYSLOG_SEPARATOR.join(pairs) | ||
|
|
||
|
|
||
| def _handle_fingerprint(module): | ||
| max_log_size = module.params["max_log_size"] | ||
| if max_log_size < 0: | ||
| module.fail_json( | ||
| msg="max_log_size must be 0 or a positive integer, got %d" % max_log_size | ||
| ) | ||
|
|
||
| fingerprint_record = _collect_fingerprint_record(module, module.params["status"]) | ||
| log_message = _format_fingerprint_syslog(fingerprint_record) | ||
|
|
||
| if module.check_mode: | ||
| module.exit_json( | ||
| result = dict( | ||
| changed=False, | ||
| message="Check mode: message not logged - [%s]" % log_message, | ||
| fingerprint=fingerprint_record, | ||
| ) | ||
| if module.params["write_log_file"]: | ||
| result["jsonl_row"] = _format_fingerprint_jsonl(fingerprint_record) | ||
| result["log_file"] = module.params["log_file"] | ||
| module.exit_json(**result) | ||
|
|
||
| module.log(log_message) | ||
|
|
||
| # we don't actually change anything, so we're not changed - writing a log message | ||
| # is not considered a change | ||
| # also, we don't want to report changed every time the role runs | ||
| module.exit_json(changed=False) | ||
| if module.params["write_log_file"]: | ||
| log_file = module.params["log_file"] | ||
| try: | ||
| _write_jsonl_log( | ||
| log_file, fingerprint_record, module.params["max_log_size"] | ||
| ) | ||
| except (IOError, OSError) as exc: | ||
| module.fail_json( | ||
| msg="Failed to write fingerprint log file %s: %s" % (log_file, exc) | ||
| ) | ||
|
|
||
| module.exit_json(changed=False, fingerprint=fingerprint_record) | ||
|
|
||
|
|
||
| def run_module(): | ||
| module_args = dict( | ||
| status=dict(type="str", required=True, choices=["begin", "success"]), | ||
| write_log_file=dict(type="bool", default=False), | ||
| log_file=dict(type="path", default="/var/log/sysroles.jsonl"), | ||
| max_log_size=dict(type="int", default=2000000), | ||
| role_name=dict(type="str", required=True), | ||
| role_path=dict(type="path", required=True), | ||
| ansible_play_hosts_all=dict(type="list", elements="str", required=True), | ||
| distribution=dict(type="str", default=""), | ||
| distribution_version=dict(type="str", default=""), | ||
|
Comment on lines
+340
to
+350
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
rg -n -C 5 \
--glob '*.yml' --glob '*.yaml' --glob '*.py' \
'(^[[:space:]]*([[:alnum:]_]+\.)?sr_fingerprint:[[:space:]]*$|^[[:space:]]*sr_message:)' \
.Repository: linux-system-roles/logging Length of output: 2807 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== relevant fingerprint module args =="
sed -n '1,125p' library/sr_fingerprint.py
sed -n '320,380p' library/sr_fingerprint.py
echo
echo "== call sites =="
sed -n '10,18p' tasks/set_vars.yml
sed -n '138,146p' tasks/main.yml
echo
echo "== structural search for free-form dict and sr_message =="
python3 - <<'PY'
import glob, re
for p in glob.glob('library/sr_fingerprint.py'):
text=open(p, encoding='utf-8').read()
print(p, 'has sr_message arg:', 'sr_message' in text, 'status required:', 'choices=["begin", "success"]' in text)
PYRepository: linux-system-roles/logging Length of output: 6620 Update the remaining
🤖 Prompt for AI Agents |
||
| ) | ||
|
|
||
| module = AnsibleModule( | ||
| argument_spec=module_args, | ||
| supports_check_mode=True, | ||
| ) | ||
|
|
||
| _handle_fingerprint(module) | ||
|
|
||
|
|
||
| def main(): | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Enforce
max_log_sizefor existing oversized logs.The trim call removes only
len(new_line). If an operator lowersmax_log_sizebelow the current file size, the next write leaves the file far above the configured limit. A new row larger thanmax_log_sizealso exceeds the limit without a defined policy.Calculate the excess as
cur_size + new_line_size - max_log_size. Remove that amount before appending. Define and test the behavior when one JSONL row exceeds the limit.🧰 Tools
🪛 ast-grep (0.45.0)
[warning] 232-232: File path is request-/variable-derived; validate and normalize to prevent path traversal.
Context: open(log_file, "a")
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').
(open-filename-from-request)
🤖 Prompt for AI Agents