tiered-storage

Real Time Big Data / IoT Machine Learning (Model Training and Inference) with HiveMQ (MQTT), TensorFlow IO and Apache Kafka - no additional data store like S3, HDFS or Spark required

A passthrough FUSE filesystem that intelligently moves files between storage tiers based on frequency of use, file age, and tier fullness.

RemoteStorageManager for Apache Kafka® Tiered Storage

Linux FUSE storage daemon with routing rules and SQLite metadata index.

An open source data tiering archive for Linux. Written in Rust. Transparently offload cold data to S3, HDD, or LTO tape while keeping searchable local stubs.

Pinterest's simplified and efficient Tiered Storage implementation for Kafka

A High-Performance Storage Infrastructure for Activity and Log Workloads

DuckDB, DuckLake & Parquet analytics at cloud scale — for .NET 10 and EF Core 10

Open research project exploring tiered storage for PostgreSQL application tables—keep active rows in Postgres, move history to Parquet, and query both through the original table.

Tiered filesystem retrieval (L0/L1/L2) with vic CLI, embeddings, optional skillware & HTTP bridge, Python reference stack.

Repository for CP Tiered Storage

Kafka wire protocol broker in a single Go binary. Consumer groups, SASL, TLS, ACLs, Prometheus, S3 tiering. Runs anywhere Docker runs.

Aether is a system that automates data placement across storage tiers (NVMe/SSD/HDD) using access patterns, ensuring high performance for hot data and optimal compression for cold data all through a unified interface.

Kafka Remote Storage implementation for Minio

Evidence-backed research, tutorials, benchmarks, and failure analysis for Windows Storage Spaces tiered storage, covering Interleave, NTFS cluster size, parity, performance, and data integrity. 基于完整实测与公开证据的 Windows Storage Spaces 分层存储研究，涵盖搭建教程、参数调优、性能测试、故障分析与数据完整性验证。

Tiered Media Storage Manager for Jellyfin - Automatically transition media between fast SSD (symlinks) and bulk HDD (MKV) storage based on watch history

The cache of the LLM's memory

Tiered distributed file system: chunked + replicated hot storage with automatic S3 cold-tier offload, transparent rehydration, self-healing replication, crash-safe metadata, and a live cluster dashboard. Pure Python.

Distributed, cloud-native Roaring Bitmaps — query and intersect billion-scale integer sets over tiered cloud storage (RAM → NoSQL → object store), at a fraction of an always-on cache.