feat(symbols): build the R8 mapping index in the CLI - #768
Merged
Conversation
`ldcli symbols upload --type android` needed a --path pointing at a directory holding mapping.txt and an --app-version matching what the app reports, so every project had to add build script code to stage the mapping somewhere and plumb its version into CI. The Android Gradle Plugin already writes both: R8's mapping at <module>/build/outputs/mapping/<variant>/mapping.txt, and the version it packaged in output-metadata.json beside the APK. Read them, and the command works from an Android project root with no flags and no build script at all. Several obfuscated variants is an error naming them rather than a guess, since only one of them is the build being shipped. An Android mapping is now also stored as mapping.txt however deep it was found. Symbolication reads it at <lane>/mapping.txt, so a mapping uploaded from a nested path was previously keyed somewhere nothing looks. Co-authored-by: Cursor <cursoragent@cursor.com>
A build that stamps a content-derived symbols id into assets/ld_symbols_id.txt had to also stage a mapping.txt.symbolsid sidecar next to a copy of the mapping, purely so the upload could be keyed by the same id the app reports. The packaged APK/AAB already carries that asset, and it is the app that will run: what it carries is exactly what will be reported. Reading it there needs nothing handed over by the build and leaves no way for the two to disagree, so the staging step goes away and a stamped build uploads on the Symbols Id Lane with no flags. An id that does not match the 32-hex-char shape the SDK reports is ignored rather than keyed on, since it names a lane nothing would ever ask for. Co-authored-by: Cursor <cursoragent@cursor.com>
A release R8 mapping is tens of megabytes of text and every build pushes it again: the e2e app's is 61.3 MB, which gzips to 5.0 MB in under half a second. React Native source maps and Apple symbol maps compress on the same order. Each object is marked Content-Encoding: gzip, so it stays self-describing — storage returns the header on read, an HTTP client inflates it in transit, and the backend inflates whatever still arrives compressed. Artifacts held in memory are compressed before an upload URL is asked for rather than at the point of sending, because the digest that proves an object is already stored is compared against the ETag of the stored bytes; hashing the artifact instead would never match and every source bundle would be re-sent. Files are compressed as they are read, streamed through the compressor so a large mapping is never held in memory whole. Anything that comes out of gzip no smaller is sent as it is, so a .srcbundle, which already gzips its own entries, is not wrapped a second time. Co-authored-by: Cursor <cursoragent@cursor.com>
… mapping An Android upload sent R8's mapping.txt as it was, tens of megabytes of text, and the backend parsed it into a random-access index on the first crash of every build that ever crashed. The mapping is here, on the machine that just produced it, so the index is built here too and the text is never stored: `upload` and `generate` now write mapping.v1.index to the Symbols Id and Version lanes. The mapping is streamed through r8index.EncodeFrom rather than parsed into memory, so a build machine does not have to find hundreds of megabytes to produce a few. A mapping that yields no index is an error: symbolication reads only the index, so storing anything else would leave a build looking like it has symbols while every crash arrives obfuscated. internal/symbols/r8index is a verbatim copy of the backend's package, the way dsymmap and srcbundle already are, and srcbundle.Builder now compresses on add so that a per-class bundle costs its compressed size rather than its raw one. Co-authored-by: Cursor <cursoragent@cursor.com>
The r8index package exists twice, here and in the backend that reads what this writes, and an index is only interchangeable if the two copies agree byte for byte. Two copies cannot import each other's tests, so what they share is a fixture: a readable mapping and the exact index bytes it encodes to, identical in both repos. Either side drifting now fails here rather than in production, where the symptom would be an unreadable index or a silently wrong frame. Co-authored-by: Cursor <cursoragent@cursor.com>
The golden fixture proves the encoder here still produces the bytes both repos agreed on; this proves the artifact a build machine wrote answers like the mapping it came from, which is the guarantee symbolication actually rests on. Opt-in via R8_MAPPING and R8_INDEX, since it needs a real release build. Co-authored-by: Cursor <cursoragent@cursor.com>
Keeps the r8index copy in step with the backend's, where reading these paths from R8_MAPPING and R8_INDEX fails a CI check that bans os.Getenv anywhere in the backend. A test flag is what the package already reaches for anyway (-update, for the golden fixture). end-of-file-fixer runs here too, through pre-commit, and it appended a newline to the backend's copy of the golden index — a byte that moves the footer the reader locates the index by, and breaks the byte-for-byte agreement the fixture exists to prove. It cannot tell a binary file from a text one, so *.index is excluded. Co-authored-by: Cursor <cursoragent@cursor.com>
#767 was squash-merged, so main carries as one commit what this branch has as three, and git saw both sides as having changed cmd/symbols/upload.go. The squash is byte-identical to what this branch started from and main has nothing else new, so the branch's own version stands and the merge changes no file. Co-authored-by: Cursor <cursoragent@cursor.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit af0a5ca. Configure here.
Generating an Android source bundle reads sourcePathFlag, which generate never registered: --source-path was rejected outright, so sources could only be scanned from the working directory. The read answered anyway, because both commands bind the same viper keys and upload's flag was left to supply the default — a value the caller of generate could neither see nor change. Also stop describing --include-sources as Apple-only, since an Android mapping carries sources now too. Co-authored-by: Cursor <cursoragent@cursor.com>
abelonogov-ld
added a commit
that referenced
this pull request
Aug 1, 2026
#768 was squash-merged, so cmd/symbols/android_upload.go arrives from both sides and git could not tell that this branch's copy is main's plus one commit: the id R8 records for a mapping, read here and used to key the upload. Main's copy is byte-identical to this branch's before that commit, so ours stands and the merge changes no file. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
symbols upload --type androidsent R8'smapping.txtas it was — 61.3 MB for the e2e app — and the backend parsed it into a random-access index on the first crash of every build that ever crashed. The mapping is here, on the machine that just produced it, so the index is built here and the text is never stored:uploadandgeneratenow writemapping.v1.indexto both the Symbols Id Lane and the Version Lane. 61.3 MB of text becomes a 5.2 MB index.Streamed, not parsed.
r8index.EncodeFromreads the mapping line by line and encodes each class as it closes, andsrcbundle.Builder.Addnow gzips an entry on add, so a per-class bundle costs its compressed size rather than its raw one. Peak live heap for that mapping is 100.3 MB, down from 407.5 MB parsing it whole — a build machine no longer has to find hundreds of megabytes to produce a few.A mapping that yields no index is an error. Symbolication reads only the index, so storing anything else would leave a build looking like it has symbols while every crash arrives obfuscated.
internal/symbols/r8indexis a verbatim copy of the backend's package, the waydsymmapandsrcbundlealready are. An index is written by one repo and read by another, so the copies are only interchangeable while they agree byte for byte: the golden fixture — a readable mapping and the exact index bytes it encodes to — is identical on both sides, so drift fails a test here rather than surfacing as an unreadable index or a silently wrong frame. The format version is in the artifact's file name, so an index written by an older CLI is a key the reader never asks for rather than bytes it has to make sense of.Android upload logic moves to
cmd/symbols/android_upload.go, and the generic upload path no longer carries Android special cases.Ordering
Stacked on #767.
The backend read side is https://github.com/launchdarkly/observability/pull/1612, and it must be deployed after this ships: it stops reading
mapping.txt, and this is what starts writing the index it reads instead. Between the two, an Android build uploads an index nothing reads yet. Any lane holding onlymapping.txtneeds re-uploading once the backend flips.Test plan
go test ./cmd/symbols/... ./internal/symbols/...R8_MAPPING/R8_INDEXcross-check: an index this CLI built answers like the 61.3 MB mapping it came frommapping.txtin the laneMade with Cursor