I intentionally kept it lightweight. Just Parquet files + simple partitioning + commits on Hugging Face. That already covers most of what I need, without introducing a heavier stack or extra dependencies.
Also, I wanted something that is easy to consume anywhere. With this setup, you can point DuckDB or Polars directly at the data and start querying, no catalog or special tooling required.
Also, I wanted something that is easy to consume anywhere. With this setup, you can point DuckDB or Polars directly at the data and start querying, no catalog or special tooling required.