Skip to content

Add a ManifestReader (symmetric to ManifestWriter) to encapsulate manifest read/decrypt/parse #2881

Description

@xanderbailey

Motivation

We have a ManifestWriter / ManifestWriterBuilder (crates/iceberg/src/spec/manifest/writer.rs) that encapsulates writing a manifest, including encryption. There is no symmetric ManifestReader for the read path.

As a result, code that needs to read a manifest from bytes has to repeat the read → decrypt (via key_metadata) → parse_avro sequence by hand. For example, in the encrypted-manifest tests added in #2821 (crates/iceberg/src/transaction/append.rs) we manually do:

let raw = table.file_io().new_input(&manifest_file.manifest_path)?.read().await?;
let key_metadata = StandardKeyMetadata::decode(key_metadata_bytes)?;
// EncryptedInputFile::new(...).read().await? then Manifest::try_from_avro_bytes(...)

ManifestFile::load_manifest (crates/iceberg/src/spec/manifest_list/manifest_file.rs:181) already implements the correct decrypt-and-parse logic, but it is only reachable when you have a ManifestFile from a manifest list. There is no standalone reader for reading a manifest directly (e.g. from a path + optional key metadata), so this logic gets duplicated.

Proposal

Introduce a ManifestReader (mirroring ManifestWriter)

Context

Follow-up from review feedback on #2821.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions