Parquet doesn't really do timestamp with time zone. Also relevantly for processi... | Hacker News

Hacker Timesnew | past | comments | ask | show | jobs | submit

		glogla on April 4, 2023 \| parent \| context \| favorite \| on: Parquet: An efficient, binary file format for tabl... Parquet doesn't really do timestamp with time zone. Also relevantly for processing Parquet, Apache Spark doesn't really do timestamp with time zone. Meaning that if your data go through parquet (e.g. in a data lake) you have to store timestamps as strings, or lose the time zone. It sucks.

noselasd on April 4, 2023 [–]

We went for the 3. option, store the timezone in a separate column for data where the timezone is needed.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact