feat: add an in-place compress path
#123Comparing
aduffy/safe-compress
(5f10dd4
) withdevelop
(d0a3601
)-17%
- Improvements
- 0
- Regressions
- 10
- Untouched
- 6
- New
- 0
- Dropped
- 0
- Ignored
- 0
Benchmarks
Failed
compress-onlyRegression
benches/compress.rs::compress_bench::bench_dbtext::dbtext/l_comment::compress-only-15%
13.1 ms
15.5 ms
train-and-compressRegression
benches/compress.rs::compress_bench::bench_dbtext::dbtext/l_comment::train-and-compress-13%
15.7 ms
18.1 ms
compress-onlyRegression
benches/compress.rs::compress_bench::bench_dbtext::dbtext/urls::compress-only-15%
34.2 ms
40.4 ms
train-and-compressRegression
benches/compress.rs::compress_bench::bench_dbtext::dbtext/urls::train-and-compress-15%
36.7 ms
42.9 ms
compress-onlyRegression
benches/compress.rs::compress_bench::bench_dbtext::dbtext/wikipedia::compress-only-16%
19.2 ms
22.8 ms
train-and-compressRegression
benches/compress.rs::compress_bench::bench_dbtext::dbtext/wikipedia::train-and-compress-14%
22.4 ms
26 ms
compressRegression
benches/micro.rs::bench_micro::bench_compress::cf=1::compress-17%
11.4 ms
13.8 ms
compressRegression
benches/micro.rs::bench_micro::bench_compress::cf=2::compress-15%
6.8 ms
8.1 ms
compressRegression
benches/micro.rs::bench_micro::bench_compress::cf=4::compress-14%
4.1 ms
4.7 ms
compressRegression
benches/micro.rs::bench_micro::bench_compress::cf=8::compress-13%
2.3 ms
2.6 ms
Passed
decompress
benches/micro.rs::bench_micro::bench_compress::cf=8::decompress0%
959.3 µs
959.2 µs
decompress
benches/compress.rs::compress_bench::bench_dbtext::dbtext/l_comment::decompress0%
5 ms
5 ms
decompress
benches/compress.rs::compress_bench::bench_dbtext::dbtext/wikipedia::decompress0%
7.3 ms
7.3 ms
decompress
benches/compress.rs::compress_bench::bench_dbtext::dbtext/urls::decompress0%
13.3 ms
13.3 ms
compress-hashtab
benches/micro.rs::bench_micro::bench_compress::compress-overhead::compress-hashtab-7%
364.7 ns
393.9 ns
compress-twobytes
benches/micro.rs::bench_micro::bench_compress::compress-overhead::compress-twobytes-10%
270.6 ns
299.7 ns
Commits
Click on a commit to change the comparison rangeBase
develop
d0a3601
-41.61%
add an in-place compress path using only safe code
Using the FSST library currently requires you to make two allocations
1. Allocate a buffer to compress_into
2. Allocate a larger packed buffer
We shouldn't take a &mut Vec<u8> directly, since Vec isn't
splittable. We should instead be taking &mut [MaybeUninit<u8>], which
can be backed by Vec, Bytes, Buffer<u8> or whatever other memory
allocation we happen to get our hands on.
This PR adds a new safe compression pathway that exposes
`compress_into_uninit` and implements the hot loop using only safe code
to boot.
Signed-off-by: Andrew Duffy <andrew@a10y.dev>
feec50c
7 days ago
by a10y+38.18%
elide bounds check (maybe?)
Signed-off-by: Andrew Duffy <andrew@a10y.dev>
01ddd72
7 days ago
by a10y-13.48%
maybe now elide bounds chekc?
Signed-off-by: Andrew Duffy <andrew@a10y.dev>
7489f0d
7 days ago
by a10y0%
fix typos
Signed-off-by: Andrew Duffy <andrew@a10y.dev>
5f10dd4
7 days ago
by a10y