Arrow is designed for zero copy ipc -- it is, by definition, an in-memory format that is therefore mmappable.
Parquet is an on-disk format, designed to be space efficient.
So for example, Parquet supports general purpose compression in addition to dictionary and RLE encodings. General purpose compression forces you to make copies, but if you're streaming from disk the extra cost of decompressing blocks is acceptable.
Arrow doesn't use general purpose compression because it would force copies to be made and dominate compute costs for data in memory.
Arrow is designed for zero copy ipc -- it is, by definition, an in-memory format that is therefore mmappable.
Parquet is an on-disk format, designed to be space efficient.
So for example, Parquet supports general purpose compression in addition to dictionary and RLE encodings. General purpose compression forces you to make copies, but if you're streaming from disk the extra cost of decompressing blocks is acceptable.
Arrow doesn't use general purpose compression because it would force copies to be made and dominate compute costs for data in memory.