Voice Support / Siri

[thiccaxe]

What to investigate?

Figure out the packet format.

Expected outcome

[thiccaxe]

Possible Candidates:
https://github.com/evandcoleman/node-appletv/blob/master/src/lib/protos/SendVoiceInputMessage.proto
https://github.com/Jack-R1/SiriRemoteVoiceDecoder/blob/master/SiriRemoteVoiceDecoder/main.m

[thiccaxe]

Threw our good "friend" claude at it and it solved the problem quick enough:

log-2026-06-20-23-45-00.txt.py contains a list of packets that contain voice data that is possibly opus encoded. The exact format is unknown. Try and recover a .wav file from it. use python if possible, and the environment is managed with uv

Let me start by reading the log file to understand the packet format.

Read c:\Users\user\Documents\CompanionGames\CompanionGames\pyatv-testing\pyatv\log-2026-06-20-23-45-00.txt.py

Now I have a better understanding of the packet structure. Let me analyze it:

Each packet has:

_i: '_siA' (some identifier)
_x: sequence number (incrementing integers)
_c: a dict with:
3: binary data (the audio payload)
4: some float (timing/bitrate info?)
5: a list of dicts with keys 1 and 2, where 2 looks like cumulative byte offsets and 1 looks like sizes
_t: 1 (type)
The key 5 seems to contain frame info:

{1: size, 2: offset} - these are frame sizes and offsets within the audio data at key 3
Looking at the first packet more closely, the frame metadata shows 5 frames totaling 42 bytes, but the actual data payload is 48 bytes, so there's some extra data not accounted for by the frame list. The second packet has a longer data section.

Now examining the frame structure more carefully - if the offset and size values define frame boundaries, then each frame in packet 1 would start at its offset position and span its size in bytes. The frames add up to 42 bytes total, but there's a 6-byte discrepancy with the actual data length. offset=18 → bytes 18-25

{1: 8, 2: 26}: size=8, offset=26 → bytes 26-33
{1: 8, 2: 34}: size=8, offset=34 → bytes 34-41
That accounts for 42 bytes total, but the actual data is 48 bytes, leaving 6 unaccounted bytes at the end. Let me verify by manually counting the hex bytes in the first frame to confirm the structure. Counting through the byte sequence character by character to verify the total length matches the expected data. So the total is 42 bytes across five frames with sizes of 9, 9, 8, 8, and 8 bytes respectively. I was initially miscounting the data length, but now I see the frame structure clearly. Let me think about what this binary format actually represents.

This looks like it's coming from the pyatv library, which handles Apple TV communication, and the message structure with fields like _i, _x, _c, and _t suggests protobuf-encoded data. The key 5 containing frame metadata with size and offset information points to audio frame data, and the _i: '_siA' identifier likely indicates a stream input audio message. The data bytes starting with 0x48 (which is 0b01001000 in binary) are characteristic of Opus codec TOC bytes that encode the codec mode, bandwidth, and frame configuration. So config 9 is SILK-only at 20ms with mono audio and a single frame per packet. Looking at the actual frame data in packet 2, I can see it contains multiple frames with their respective sizes and byte offsets—an 8-byte frame at the start, another 8-byte frame, then a 21-byte frame, and finally a 37-byte frame.

Each of these frames is an Opus packet with its own TOC byte (0x48), so to decode this I'll need to extract each frame from the binary data and feed them individually to an Opus decoder, then write the decoded audio to a WAV file.

Looking at the packet structure, the _x counter increments sequentially and each packet contains around 4-5 frames. If each frame represents 20ms of audio at 8kHz or 16kHz, the total duration across all 17 packets would be roughly 1.5 seconds of audio.

I'm going to write a Python script that parses the packet data, extracts the individual Opus frames, decodes them using opuslib, and outputs the result as a WAV file.

Examining the frame boundaries more closely, I need to understand whether the TOC byte is already included in each frame or needs to be prepended separately.

Looking at the extracted frames, each one starts with 0x48, which is the Opus TOC byte indicating SILK narrowband at 20ms with mono audio and a single frame per packet. So the frames in the data already contain their TOC bytes built in.

Now I'm writing a Python script to extract and decode these Opus frames into a WAV file, using the opuslib decoder configured for 8kHz mono based on what the TOC byte tells us. Finishing the WAV file configuration and then realizing I need to double-check the frame_info structure — it looks like each entry uses protobuf field numbers where key 1 is the size and key 2 is the offset.

Now I'm deciding between importing the log file directly or using a standalone script, but the filename with hyphens makes importing tricky, so I'll probably use importlib.util or just execute the file to load the data. I'm also starting to analyze the Opus TOC byte to understand the encoding format.

Looking at the TOC structure, the high 5 bits give me the config (9 = SILK narrowband at 8000 Hz), bit 2 indicates mono, and the low 2 bits show it's a single frame. For decoding with opuslib, I need to set up a decoder for 8000 Hz mono and decode 20ms frames, which translates to 160 samples per frame at that sample rate.

The _c[4] values don't seem to be timing information—the first packet shows essentially 1 microsecond while later packets have much larger values like 0.045, suggesting this might be a buffer level or quality metric instead. Rather than digging deeper into the protocol, I should just write the decoding script and test it. Looking at the packet sequence, each one contains between 4-5 frames of audio data.

Writing the Python script...

Let me also look at the atvproxy.py and check if there's any related code in pyatv that explains the packet format.

Grep "_siA|streamInput|voice|audio|opus" (in c:\Users\user\Documents\CompanionGames\CompanionGames\pyatv-testing\pyatv, type: py)

Let me look at the companion protocol to understand the _siA message type and the packet structure.

Grep "_siA|siA" (in c:\Users\user\Documents\CompanionGames\CompanionGames\pyatv-testing\pyatv, type: py)

Good - I can see from the log file log-2026-06-20-23-45-10.txt that these are Companion protocol OPACK messages. The _i: '_siA' means "stream input Audio" likely. The message has:

_i: '_siA' - message identifier (streamInputAudio?)
_x: sequence number
_c: content with:
3: raw audio data (Opus frames concatenated)
4: some float value (buffer level?)
5: list of frame info dicts with {1: size, 2: offset}
_t: 1 - timestamp field
Let me check the companion protocol code to understand what these fields represent.

Let me look at the companion protocol and check for any related audio handling code.
Read c:\Users\user\Documents\CompanionGames\CompanionGames\pyatv-testing\pyatv\pyatv\protocols\companion_init_.py
Glob pattern: "**/*.proto"

Let me look at the SendVoiceInputMessage proto to understand the format better:

Read c:\Users\user\Documents\CompanionGames\CompanionGames\pyatv-testing\pyatv\pyatv\protocols\mrp\protobuf\SendVoiceInputMessage.proto
Read c:\Users\user\Documents\CompanionGames\CompanionGames\pyatv-testing\pyatv\pyatv\protocols\mrp\protobuf\VoiceInputDeviceDescriptorMessage.proto
Read c:\Users\user\Documents\CompanionGames\CompanionGames\pyatv-testing\pyatv\pyatv\protocols\mrp\protobuf\AudioFormatSettingsMessage.proto

Now let me look at the Companion protocol's audio handling. The _siA seems to be a Companion-specific OPACK message. Let me look at the companion API code.

Glob pattern: "**/*.py"
Grep "siA|voice|audio|stream.*input" (in c:\Users\user\Documents\CompanionGames\CompanionGames\pyatv-testing\pyatv\pyatv\protocols\companion\api.py)

Let me look at the atvproxy.py more carefully to understand how the _siA messages are logged, and also let me look at the full log text file to understand what happened during the session.

got too lazy to format this nicely

Now I understand the format. This is from the Apple Companion protocol, where the _siA (stream input Audio) messages contain Opus-encoded audio frames. The structure is:

_c[3]: raw audio data buffer containing all Opus frames concatenated
_c[4]: buffer level/quality metric (float)
_c[5]: list of {1: size, 2: offset} dicts describing individual Opus frame positions within the buffer
The Opus TOC byte 0x48 (01001000) indicates:

Config 9 (SILK NB, 20ms) → 8000 Hz, mono
Single frame per packet
I'll write a Python script using opuslib to decode these frames, though I need to ensure libopus is available on Windows.

The TOC byte 0x48 breaks down to 8kHz mono with 20ms frames, which means each frame contains 160 samples at 8kHz. For decoding, I'll instantiate a decoder with those parameters and pass the frame data along with the sample count.

Since opuslib requires system-level libopus installation, I should consider fallback approaches like pyogg or piping the frames through ffmpeg if the primary method doesn't work in the uv environment.

Looking more closely at the packet structure, I need to examine how the frame data is actually extracted from the _c[5] field containing packet descriptions. use ffmpeg as a fallback. Actually, the most reliable approach is to construct a proper Ogg Opus file with the required header pages, then decode that using ffmpeg or a library that can handle the container format. This way I can ensure the frames are properly wrapped in the Ogg structure that decoders expect. Writing the Python script... Now I'm extracting the Opus frames from each packet using the frame descriptors to locate them in the raw data, then printing out details about the first few frames to verify they have the expected TOC byte. After that, I'm attempting to import opuslib to decode them.

Setting up the decoder with 8kHz sample rate and mono audio, then iterating through each frame to decode it into PCM samples, catching any errors that occur during decoding. Once decoded, I'm writing all the PCM frames to a WAV file with the proper format settings. If opuslib isn't available, I'm falling back to writing the raw frames to a binary file with instructions for manual conversion using ffmpeg.

I'm realizing that opuslib on Windows might have issues finding the required DLL, so I should consider using uv to install it properly. I'm weighing a few practical approaches here: using opuslib directly if the DLL dependency can be resolved, leveraging pydub with ffmpeg as a fallback, or manually wrapping the Opus frames into an Ogg container format that standard players can handle. The manual Ogg encoding is self-contained but requires careful implementation of the container structure, so I'll sketch out a solution that tries the simpler methods first before falling back to that more complex approach. Each Ogg page contains a segment table specifying the size of each data chunk, followed by the concatenated segment data. The OpusHead header identifies the codec with its magic string and specifies the channel count, pre-skip samples (312 for 48kHz), input sample rate, output gain in Q7.8 format, and channel mapping family. The OpusTags header follows with a vendor string and user comment list.

For audio pages, the granule position tracks total PCM samples decoded at 48kHz regardless of the actual encoding sample rate. With 8kHz SILK NB at 20ms frames, each frame represents 160 samples at 8kHz, which translates to 960 samples when counted at the standard 48kHz granule position rate.

I'm reconsidering the approach here—writing directly to OGG Opus format would work, but the user specifically asked for a WAV file. I could either decode the Opus frames to PCM using opuslib and then write WAV, or generate OGG Opus and rely on ffmpeg being available. Let me check if I can use opuslib through uv to handle the decoding directly. 's a pyproject.toml file I can reference to see what dependencies are already available.

Let me check the existing project structure and what packages are available.

Globpattern: "pyproject.toml"
Found 1 file
Bash Check if ffmpeg is available
IN
where ffmpeg 2>/dev/null && echo "ffmpeg found" || echo "ffmpeg not found"

OUT
C:\Users\user\AppData\Local\Microsoft\WinGet\Packages\FFmpeg.GPL_Microsoft.Winget.Source...\ffmpeg.exe
ffmpeg found
ffmpeg is available. Now let me look at the pyproject.toml to understand the project setup.

ffmpeg is available. Now I have everything I need. The approach:

Extract Opus frames from the packets (each frame is at _c[3][offset:offset+size] per _c[5])
Wrap them in a minimal Ogg Opus container (no external deps needed)
Use ffmpeg to convert to WAV
The TOC byte 0x48 = SILK NB, 8kHz, 20ms, mono. Let me write the script now.

Write c:\Users\user\Documents\CompanionGames\CompanionGames\pyatv-testing\pyatv\decode_voice.py
223 lines
#!/usr/bin/env python3
"""
Recover a WAV file from pyatv Companion protocol voice packets.

Packet format (_siA = stream input Audio):
_c[3] : raw buffer containing concatenated Opus frames
_c[4] : buffer level (float, informational)
_c[5] : list of {1: frame_size_bytes, 2: byte_offset} frame descriptors
_x : sequence number (monotonically increasing)

TOC byte 0x48 = 0b01001000:
config 9 = SILK-only NB (8 kHz), 20 ms frame, mono (s=0), code 0 (1 frame)
=> each Opus frame is 20 ms of 8 kHz mono audio

Strategy: pack Opus frames into an Ogg Opus bitstream, then decode to WAV
with the bundled ffmpeg (no extra Python dependencies required).

code

if __name__ == "__main__":
    main()
Bash Run the decode script with uv
IN
cd "c:\Users\user\Documents\CompanionGames\CompanionGames\pyatv-testing\pyatv" && uv run python decode_voice.py 2>&1

OUT
Exit code 1
ffmpeg stderr: ffmpeg version N-versionCopyright (c) 2000-2025 the FFmpeg developers

  built with gcc version (crosstool-NG version)

  configuration: --config-flags

  lib libversion

[in#0 @ memory_address] Error opening input: End of file

Error opening input file pipe:0.

Error opening input files: End of file


b'this contains my voice in it'
Extracted 81 Opus frames
  Frame 0: 9 bytes, TOC=0x48 config=9 stereo=0 code=0
  Frame 1: 9 bytes, TOC=0x48 config=9 stereo=0 code=0
  Frame 2: 8 bytes, TOC=0x48 config=9 stereo=0 code=0
Built Ogg Opus stream (5560 bytes)
Traceback (most recent call last):
  File "C:\Users\user\Documents\CompanionGames\CompanionGames\pyatv-testing\pyatv\decode_voice.py", line 222, in <module>
    main()
    ~~~~^^
  File "C:\Users\user\Documents\CompanionGames\CompanionGames\pyatv-testing\pyatv\decode_voice.py", line 215, in main
    decode_to_wav(ogg_data, out_wav)
    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user\Documents\CompanionGames\CompanionGames\pyatv-testing\pyatv\decode_voice.py", line 190, in decode_to_wav
    raise RuntimeError(f"ffmpeg failed with code {result.returncode}")
RuntimeError: ffmpeg failed with code 3753488571
The ffmpeg error is "End of file" when reading from stdin. This is strange - maybe the issue is with Windows and piped input? Let me try writing the Ogg file to disk first and then running ffmpeg on it.

Also, the output shows that the frames extracted from the packets (printed as hex by the existing code at the bottom of the log file) look correct. 81 frames total.

Let me also check if there's an issue with my Ogg page format. The "End of file" error from ffmpeg suggests that either:

The Ogg file is malformed
There's an issue with stdin piping on Windows I'll write the Ogg data to a temporary file instead of piping it, and I need to suppress the hex output that's being printed from the log file's for loop when it gets imported. Now I'm verifying the Ogg page header structure and checking my struct format against the specification—the header type uses individual bit flags for continuation, beginning of stream, and end of stream markers, and my packing format should account for all the fields correctly, totaling 27 bytes before the segment table. I'm checking the byte positions against the actual Ogg format specification to verify my struct layout is correct.
Fixing the struct definition... I'm realizing the struct format is wrong—I packed the checksum as a single byte instead of 4 bytes, which throws off the entire header layout. Looking at the format string "<4sBBqIIB", that last B should actually be an I for the 4-byte checksum field, and then the segment table gets appended separately. Building the Ogg page header... I'm realizing the Opus frames in this data are actually quite small, typically under 126 bytes, so they'll always fit in a single segment without needing the continuation logic I was overthinking. Let me verify the actual frame sizes to confirm this approach works. Looking at the hex output, some frames appear to be 60+ bytes, which is still well under the 255-byte limit. The main issue I need to fix is the missing checksum field in my struct definition, and I should verify my Ogg CRC implementation is correct—it uses a specific polynomial (0x04c11db7) rather than the standard zlib CRC32. I also notice I'm importing zlib unnecessarily and need to clean that up. For the segment table construction, I was building it by chunking data into 255-byte segments. I'm wondering if ffmpeg failed due to the Ogg format being invalid or if it's a Windows stdin piping issue. The error message shows ffmpeg couldn't read any data from stdin at all, which suggests either the file is corrupt or there's a subprocess communication problem on Windows. I should fix the Ogg page builder first, then try writing to a temporary file instead of piping to avoid potential stdin issues. I also notice the debug output from the log file's for loop is going to stdout, but that shouldn't be affecting ffmpeg's input. I think the issue might be that subprocess.run with input=data writes everything synchronously before returning, so a race condition is unlikely. Let me try writing to a temporary file instead and have ffmpeg read from that, which would also let me verify if there's a Windows-specific problem with reading from pipe:0. I should also add some debugging to check if the Ogg page structure is correct, and fix that struct bug while I'm at it.

The Ogg container has a struct bug (missing the 4-byte CRC field) and Windows stdin piping to ffmpeg is failing. Let me fix both — write to a temp file and fix the page format.

Edit c:\Users\user\Documents\CompanionGames\CompanionGames\pyatv-testing\pyatv\decode_voice.py
Modified
Let me fix the unused import and fix the main bugs. Let me read the current state of the file.

Read c:\Users\user\Documents\CompanionGames\CompanionGames\pyatv-testing\pyatv\decode_voice.py (lines 91-220)
Now I can see the bugs clearly:

The struct "<4sBBqIIB" doesn't include the 4-byte CRC placeholder. The correct format should be "<4sBBqIII" (three unsigned ints: serial, seq, checksum), then separately add num_segs.

The lacing table also has bugs: segments.append(b'\xff') should append an integer 255, not bytes.

The checksum is patched at offset 22, but with the wrong struct layout, the actual offset of the checksum will be wrong.

There's also a potential issue with ffmpeg reading from stdin on Windows that could cause problems.

I need to rewrite both _make_page and decode_to_wav functions. For _make_page, I'm building the lacing values for the segment table by breaking the data into chunks, where each segment under 255 bytes gets its own entry.

Rewriting the functions... Writing Ogg page structure... Writing decode function...

I'll also suppress the stdout during module loading by redirecting it to a StringIO buffer while executing the module, then restoring it afterward. Now let me put together the complete corrected file.

Now I'll rewrite the file to fix: the missing 4-byte CRC slot in the struct, the broken lacing table, and Windows stdin piping.

Write c:\Users\user\Documents\CompanionGames\CompanionGames\pyatv-testing\pyatv\decode_voice.py
228 lines
#!/usr/bin/env python3
"""
Recover a WAV file from pyatv Companion protocol voice packets.

Packet format (_siA = stream input Audio):
  _c[3]  : raw buffer containing concatenated Opus frames
  _c[4]  : buffer level (float, informational)
  _c[5]  : list of {1: frame_size_bytes, 2: byte_offset} frame descriptors
  _x     : sequence number (monotonically increasing)

TOC byte 0x48 = 0b01001000:
  config 9 = SILK-only NB (8 kHz), 20 ms frame, mono (s=0), code 0 (1 frame)
  => each Opus frame is 20 ms of 8 kHz mono audio

Strategy: pack Opus frames into an Ogg Opus bitstream file, then decode to
WAV with ffmpeg (no extra Python dependencies required).
"""
code
"""
Bash Run the fixed decode script
IN
cd "c:\Users\user\Documents\CompanionGames\CompanionGames\pyatv-testing\pyatv" && uv run python decode_voice.py 2>&1

OUT
Extracted 81 Opus frames (1.62 s of audio)
  Frame 0: 9 bytes, TOC=0x48 (config=9, stereo=0, code=0)
  Frame 1: 9 bytes, TOC=0x48 (config=9, stereo=0, code=0)
  Frame 2: 8 bytes, TOC=0x48 (config=9, stereo=0, code=0)
Built Ogg Opus stream: 5809 bytes
Output: C:\Users\user\Documents\CompanionGames\CompanionGames\pyatv-testing\pyatv\voice_output.wav  (50.5 KB)
It worked. The script extracted 81 Opus frames (1.62 seconds of audio) and decoded them to voice_output.wav (50.5 KB).

What the format turned out to be:

Apple Companion protocol OPACK messages (_siA = stream input Audio)
_c[3]: raw buffer of concatenated Opus frames
_c[5]: frame descriptors as {1: size_bytes, 2: byte_offset} — each frame has its own Opus TOC byte (0x48) indicating SILK narrowband, 8 kHz, 20 ms, mono
_c[4]: buffer level float (informational, ignored for decoding)
What the script does:

Loads packets from the log, sorts by sequence number (_x), extracts each Opus frame using the offset/size table
Wraps all frames in a minimal Ogg Opus container (RFC 7845/3533) written in pure Python — no extra deps needed
Calls ffmpeg to decode the .ogg → .wav at 16 kHz

[thiccaxe]

The question now is how to actually deal with a .wav -> opack over the connection. It WOULD be nice if we could just send a huge opack message with many packets,

but since the packet boundary data is at the end of packet and not throughout, it wouldn't be able to start parsing the initial packets.

I'll try a large file tomorrow all at once, but worst comes to worst I think there is already some framework in pyatv to stream a file over airplay, which might be able to be adapated.

[thiccaxe]

Apparently it's very easy to soft lock the apple tv by sending over the voice packets incorrectly

[thiccaxe]

The current status is that just replaying the packets (button press for siri, siri start, siA, siri stop) is not enough - the siri icon opens, and shows movement indicating audio is detected, but the request isn't actually processed. (text isn't transcribed)

Not sure where to progress from here.

[thiccaxe]

My current hypothesis is that apple has some sort of feature flag that is sent by the device in the systeminfo stage. Basically, if siri is disabled on the remote idevice, it won't allow voice data to be sent to the apple tv. So the UI interaction is tied to the voice packets. But the actual processing on the voice data is tied to siri being enabled. In normal operation, voice will only get sent if siri is enabled anyway

It could also have something to do with icloud? Siri processing has SOMETHING to do with icloud accounts, see https://support.apple.com/en-us/105019

Action plan, hopefully one of these solves the issue:
(1) disable siri on remote device, see if apple tv siri still allowed -> still allowed
(2) intercept and block systeminfo packet (where siri related content is present) from being sent in atvproxy -> does this result in same behavior as now -> siri still works, but this could be due to improper state management
(3) hack in a system info packet into companion protocol -> does not work with random IDs. Who knows where these IDs come from

If none of these work, a decent tool that will be useful:
(4) dump an atvproxy companion session and allow it to be replayed later. Need to be intelligent about this, since requests from the apple tv or events from the apple tv do not necesarilly come in the same order -> maybe have a look up table of apple tv -> remote requests, and if all responses are the same, send that same response every time in the replay?

Looks like (4) is going to be required. Probably something to make claude do.

[thiccaxe]

Deleting the siri info from the systeminfo packet through the proxy softlocked siri on the apple tv

[thiccaxe]

Interesting, even after a restart of the apple tv, siri continued to not work from the remote. I suspected repairing may be needed. However this softlocked the remote app, so I had to restart the iphone, after which it repaired after some persusation.

It turns out that disabling siri actually DID disable the voice capability of the remote app, and after re-enabling siri on the remote device, siri started working again on the apple tv.

This means that there isn't any linked to repairing as far as siri off -> siri on goes, so pyatv users shouldn't need to repair assuming this goes through

[thiccaxe]

Claude cooked up some sort of replay format, it clearly has no clue about any of the semantics of the companion protocol state, but at least its gotten to the point where the siri UI puts up a sign that says "hmm, I'm having trouble hearing you."

[thiccaxe]

It seems that the audio routing is gated on some sort of apple id/authentication check. My only avenue is using an:

• iphone that is not signed into icloud;
• and/or an apple tv not signed into icloud
  completely isolated from the network to see if siri is still possible at all.

If that's the case, at least something should be possible since clearly apple blessing is not needed. If not, probably no way forward for this issue.

[thiccaxe]

I don't really want to factory reset my apple tv (that's the only way afaik to clear the internal store of devices) so if someone else probably with a lot of appletv's / a spare idevice wants to take a stab at this (maybe @postlund has a lot?)