Skip to content

Do not let the console encoding decide the exit code - #3

Merged
sotashimozono merged 1 commit into
mainfrom
fix/stdout-encoding
Aug 8, 2026
Merged

Do not let the console encoding decide the exit code#3
sotashimozono merged 1 commit into
mainfrom
fix/stdout-encoding

Conversation

@sotashimozono

Copy link
Copy Markdown
Member

The bug

The final print(report) writes to stdout under the platform default encoding, while the report contains em dashes and status emoji. On a console that cannot encode them the print raises UnicodeEncodeError, the script dies, and the gate exits non-zero even when every reference resolved — a clean bibliography is reported as a failed check.

Found on a Japanese Windows shell (cp932) while evaluating this action for a LaTeX paper repository. PYTHONIOENCODING=ascii reproduces it on any platform:

$ PYTHONIOENCODING=ascii python verify_references_gate.py --allow none.allow --report r.md < clean.jsonl
Traceback (most recent call last):
  File "verify_references_gate.py", line 216, in main
    print(report)
UnicodeEncodeError: 'ascii' codec can't encode character '—' in position 52
$ echo $?
1        # every reference was valid

CI is unaffected (Linux, UTF-8), so this only bites people running the gate locally — exactly when they are debugging a bibliography.

The fix

--report and --github-output already open with encoding="utf-8"; stdout now gets the same treatment: reconfigure to UTF-8 with errors="replace", plus a lossy fallback if stdout cannot be reconfigured (replaced by a StringIO in tests, for instance). The exit code then always reflects the reference check rather than the terminal. Report content is unchanged, and under ascii stdout it is still printed in full UTF-8.

Test

The existing smoke job is Linux/UTF-8 only and could not have caught this, so the PR adds an encoding job that runs the gate under ascii, cp932 and utf-8 stdout and asserts both directions — a clean record exits 0 with no traceback, a broken record still exits 1. Verified locally before pushing:

ascii    clean_exit=0 (want 0)  broken_exit=1 (want 1)  traceback=0
cp932    clean_exit=0 (want 0)  broken_exit=1 (want 1)  traceback=0
utf-8    clean_exit=0 (want 0)  broken_exit=1 (want 1)  traceback=0

🤖 Generated with Claude Code

The Markdown report contains em dashes and status emoji, and the final
print() sent it to stdout under the platform default encoding. On a
console that cannot represent those characters the print raised
UnicodeEncodeError, so the gate exited non-zero with a traceback even
when every reference resolved -- a clean bibliography reported as a
failed check.

Seen on a Japanese Windows shell (cp932) while evaluating the action for
a LaTeX repository; PYTHONIOENCODING=ascii reproduces it anywhere. The
report and GITHUB_OUTPUT files already opened with encoding="utf-8"; this
gives stdout the same treatment (reconfigure to UTF-8 with
errors="replace", plus a lossy fallback if stdout cannot be
reconfigured), so the exit code always reflects the reference check
rather than the terminal.

Adds a CI job that runs the gate under ascii / cp932 / utf-8 stdout and
asserts both directions: a clean record exits 0 with no traceback, a
broken record still exits 1. The existing Linux-only smoke test could
not have caught this.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@sotashimozono
sotashimozono merged commit c51a3fd into main Aug 8, 2026
2 checks passed
@sotashimozono
sotashimozono deleted the fix/stdout-encoding branch August 8, 2026 16:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant