Implement Pluggable Serialization Layer (Phase 3)

[biosynthart]

Goal

Create the pluggable serialization infrastructure for effects. JSON is the default (WebSocket-compatible), with stubs for msgpack and protobuf for future performance needs.

Files to Create/Modify

• ecosim/serialization/__init__.py (NEW) — SerializerRegistry export
• ecosim/serialization/interface.py (NEW) — Serializer ABC + SerializerRegistry
• ecosim/serialization/json_serializer.py (NEW) — JsonSerializer implementation

Deliverables

1. Serializer ABC (interface.py)

class Serializer(ABC):
    @abstractmethod
    def serialize(self, effect: Effect) -> bytes | str: ...

    @abstractmethod
    def deserialize(self, data: bytes | str) -> Effect: ...

    @abstractmethod
    def serialize_batch(self, effects: list[Effect]) -> bytes | str: ...

    @abstractmethod
    def deserialize_batch(self, data: bytes | str) -> list[Effect]: ...

class SerializerRegistry:
    def register(self, name: str, serializer: Serializer) -> None
    def get_default(self) -> Serializer
    def get(self, name: str) -> Serializer

2. JsonSerializer (json_serializer.py)

Default implementation using Effect.to_dict(). Compatible with existing WebSocket protocol and browser-based replay tools.

3. EffectLog Class (effects.py — add to existing file)

Append-only log of all effects for deterministic replay:

class EffectLog:
    def append(self, tick: int, effects: list[Effect]) -> None
    def get_tick_effects(self, tick: int) -> list[Effect] | None
    def serialize_log(self, start_tick: int = 0, end_tick: int | None = None) -> bytes | str

4. Engine Integration

Engine holds a reference to the serializer registry and uses it for effect logging:

class EcosystemEngine:
    def __init__(self):
        self.serializer_registry = SerializerRegistry()
        self.serializer_registry.register("json", JsonSerializer())
        self.effect_log = EffectLog(serializer=self.serializer_registry.get_default())

    def step(self, dt=0.1):
        ...
        # After applying effects:
        self.effect_log.append(self.tick, all_effects)

Unit Tests Required (tests/test_serialization.py)

1. test_json_serializer_roundtrip_single_effect — Serialize StateVarDelta → deserialize → verify equality
2. test_json_serializer_roundtrip_batch — Serialize list of effects → deserialize → verify all match
3. test_serializer_registry_registers_and_retrieves — Register "json" and "msgpack", retrieve by name
4. test_effect_log_appends_and_retrieves_by_tick — Append effects for ticks 1-10, retrieve tick 5
5. test_effect_log_serializes_to_json_string — serialize_log() returns valid JSON string

Verification Criteria

• JsonSerializer produces output identical to current WebSocket packet format (for events)
• SerializerRegistry allows easy addition of msgpack/protobuf implementations later
• EffectLog can serialize a full tick's effects and deserialize them back without loss

[biosynthart]

⏸️ Shelved — pluggable serialization layer deferred until after the distributed engine scalability work. Serialization will be needed for cross-engine message passing, but the protocol should be designed around the actual distributed architecture rather than in isolation.