Compression helpers for BEVE and parallel compression #857
stephenberry
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
BEVE is designed to be easily compressed and therefore significantly benefits from compression; both for data storage and for network use. Adding some helpers for serializing and compressing would be useful. But, even more beneficial would be a parallel compression algorithm that runs at the same time as BEVE serialization. Much like the proposed array, object, value delineation for streaming, I'd like to use this delineation for allowing the compression algorithm run to a serialization point. Typically the BEVE serialization will run much faster than the compression algorithm, but we need to add thread_safe logic to ensure our compression algorithm does not overtake the serialization.
Beta Was this translation helpful? Give feedback.
All reactions