Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upBrotli for BinaryEncoding compression #206
Conversation
I received a suggestion to support Brotli, and the rationale is pretty solid: > The WOFF 2.0 team dropped both gzip and lzma to power their latest system with brotli. Because of that, brotli has already found its way to chrome and android. http://www.w3.org/TR/WOFF20ER/ > > Brotli can compress 5-25 % more than gzip, is a little bit more efficient for small files than LZMA, and decompresses 3-5x faster than LZMA. (Brotli typically compresses less than LZMA for large files -- above 1 MB or so.) > > https://github.com/google/brotli > > https://datatracker.ietf.org/doc/draft-alakuijala-brotli/ > > Once the specifics for WebAssembly are clear, we could even specialize the context map or the static dictionary for WebAssembly (or you can design WebAssembly it in a way that it takes maximum benefit of the existing brotli context modes -- there are three binary modes and an utf-8 mode).
|
Brotli might make sense if the WebAssembly were going to specify a form of generic compression, but so far we've decided against that. Perhaps one day we'll get a builtin stream transformer (from the Streams API) so Brotli will be an attractive option for that reason. Otherwise, I'd at least include lzma and lzham in the list of alternatives. |
|
@lukewagner agreed, I added LZHAM. |
|
lgtm |
Brotli for BinaryEncoding compression
jedie
commented
Jun 22, 2015
|
I also look for a good compressor. The starting point is: pypyjs/pypyjs.github.io#4 It's not easy to find a compressor. Because there are different requirements.
The main question is: What's fast enough?!? There are two good resources for compression algorithm: Here a chart from the first link with:
You can see, in this case brotli is nice. But lzham is faster and has a better ratio... I don't know if lzham will be fast enough. There are some more interesting compressors. You will find some interesting at the second link: http://mattmahoney.net/dc/text.html |
jfbastien commentedJun 19, 2015
I received a suggestion to support Brotli, and the rationale is pretty solid: