Git.__call__ ends with a bare output.decode():
https://github.com/simplistix/giterator/blob/master/giterator/git.py
Git never transcodes blob content — git log -p / git diff splice raw file bytes into their output, so UTF-8 is a convention, not a guarantee. Any repo whose history contains a latin-1/cp1252 file (a legacy-encoding test fixture, an Excel-exported CSV, ...) makes every giterator call that sweeps such a commit blow up:
File ".../giterator/git.py", line 93, in __call__
return output.decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf6 in position 43165467: invalid start byte
Hit in the wild running chimera's is_merged (which does git log -p <mb>..<base>) against a large work monorepo — one commit somewhere in the range contains a 0xf6 (ö in latin-1) and the whole command dies.
The error path two lines up has the same latent bug: e.output.decode() inside the GitError message formatting will raise UnicodeDecodeError instead of the intended GitError when a failing command emits non-UTF-8 output.
Suggested fix: decode with errors='replace' (or surrogateescape, which round-trips) in both places. Possibly also worth offering a bytes-out escape hatch for callers that pipe git output onward (e.g. into git patch-id) and never need it as text, but the errors= change alone stops the crashes.
Git.__call__ends with a bareoutput.decode():https://github.com/simplistix/giterator/blob/master/giterator/git.py
Git never transcodes blob content —
git log -p/git diffsplice raw file bytes into their output, so UTF-8 is a convention, not a guarantee. Any repo whose history contains a latin-1/cp1252 file (a legacy-encoding test fixture, an Excel-exported CSV, ...) makes every giterator call that sweeps such a commit blow up:Hit in the wild running chimera's
is_merged(which doesgit log -p <mb>..<base>) against a large work monorepo — one commit somewhere in the range contains a0xf6(öin latin-1) and the whole command dies.The error path two lines up has the same latent bug:
e.output.decode()inside theGitErrormessage formatting will raiseUnicodeDecodeErrorinstead of the intendedGitErrorwhen a failing command emits non-UTF-8 output.Suggested fix: decode with
errors='replace'(orsurrogateescape, which round-trips) in both places. Possibly also worth offering a bytes-out escape hatch for callers that pipe git output onward (e.g. intogit patch-id) and never need it as text, but theerrors=change alone stops the crashes.