fix: bound zip archiver standard error collection - #324
Conversation
|
Hi @MarshallOfSound — this bounds unbounded stderr collection in SQRLZipArchiver and adds a regression test for excessive diagnostic output. Since you recently updated this file in #308, could you please take a look when you have a chance? |
MarshallOfSound
left a comment
There was a problem hiding this comment.
Verified locally, SQRLZipArchiverSpec all green including the new test. Nice — factory variant of aggregate is the right call given the repeat, and killing the readabilityHandler on EOF fixes the post-EOF spin as a bonus. Few things inline.
|
@MarshallOfSound Thanks again for the review — I have addressed all of the inline suggestions in commit 0a8ae53.
script/test -derivedDataPath DerivedData passes (88 tests). Could you please take another look when you have a chance? |
|
@MarshallOfSound I wanted to get your take on a simpler alternative for the UTF-8 handling before changing the current implementation. Instead of finding a UTF-8 boundary while accumulating stderr, we could keep the original bounded NSMutableData accumulator unchanged. At the final decode point, attempt UTF-8 decoding normally; only if it returns nil, retry after removing 1, then 2, then 3 bytes from the end. Because the cap only slices the tail of the byte stream, any new invalid UTF-8 introduced by the cap should be confined to that suffix; a UTF-8 scalar is at most 4 bytes. This keeps the 1 MiB cap behavior unchanged and avoids the byte-mask helper plus the extra truncated state in the accumulator. The retry would only run once on the task-failure path. I realize your inline suggestion preferred boundary handling over a decode-retry loop, so I have not changed the implementation to this approach. Would you be open to this tradeoff, or would you prefer to keep the current boundary-at-truncation approach? |
Summary
RACSignal collectstderr accumulation with a streamingaggregate that retains at most 1 MiB per
dittoinvocation.verifies that the retained error text is capped.
Root cause
SQRLZipArchiverforwards everyNSFileHandle.availableDatacallback into aRACSubject.collectretains every chunk in anNSMutableArray, after whichthe code concatenates all chunks into another
NSMutableData. A malformedarchive can make
dittowrite excessive diagnostics and cause an unboundedallocation in the host process.
Observed crash path (root-cause-relevant frames)
PartitionExcessiveAllocationSizeis a deliberate PartitionAlloc terminationwhen one allocation exceeds the direct-map limit (roughly 2 GiB), rather than a
generic low-memory termination. Here the unbounded
collectarray is theallocation owner; the Foundation frame is where that array growth reaches the
allocator.
The error output is used only to populate
NSLocalizedDescriptionwhen thedittotask exits unsuccessfully. Exit status and normal archive behavior areunchanged. The first 1 MiB of diagnostics is retained; later data is still read
and discarded so the child process cannot block on a full stderr pipe.