aduffy/safe-compress - Branch - spiraldb/fsst

feat: add an in-place compress path

#123

Comparing

aduffy/safe-compress

(

5f10dd4

) with

develop

(

d0a3601

)

-17%

Regressions: 10

Untouched: 6

Benchmarks

Failed

compress-only

benches/compress.rs::compress_bench::bench_dbtext::dbtext/l_comment

Regression

-15%

13.1 ms15.5 ms

train-and-compress

benches/compress.rs::compress_bench::bench_dbtext::dbtext/l_comment

Regression

-13%

15.7 ms18.1 ms

train-and-compress

benches/compress.rs::compress_bench::bench_dbtext::dbtext/urls

Regression

-15%

36.7 ms42.9 ms

compress-only

benches/compress.rs::compress_bench::bench_dbtext::dbtext/urls

Regression

-15%

34.2 ms40.4 ms

compress

benches/micro.rs::bench_micro::bench_compress::cf=1

Regression

-17%

11.4 ms13.8 ms

compress

benches/micro.rs::bench_micro::bench_compress::cf=2

Regression

-15%

6.8 ms8.1 ms

compress

benches/micro.rs::bench_micro::bench_compress::cf=4

Regression

-14%

4.1 ms4.7 ms

compress

benches/micro.rs::bench_micro::bench_compress::cf=8

Regression

-13%

2.3 ms2.6 ms

train-and-compress

benches/compress.rs::compress_bench::bench_dbtext::dbtext/wikipedia

Regression

-14%

22.4 ms26 ms

compress-only

benches/compress.rs::compress_bench::bench_dbtext::dbtext/wikipedia

Regression

-16%

19.2 ms22.8 ms

Passed

decompress

benches/micro.rs::bench_micro::bench_compress::cf=8

959.3 µs959.2 µs

decompress

benches/compress.rs::compress_bench::bench_dbtext::dbtext/l_comment

5 ms5 ms

decompress

benches/compress.rs::compress_bench::bench_dbtext::dbtext/wikipedia

7.3 ms7.3 ms

decompress

benches/compress.rs::compress_bench::bench_dbtext::dbtext/urls

13.3 ms13.3 ms

compress-hashtab

benches/micro.rs::bench_micro::bench_compress::compress-overhead

-7%

364.7 ns393.9 ns

compress-twobytes

benches/micro.rs::bench_micro::bench_compress::compress-overhead

-10%

270.6 ns299.7 ns

Commits

Click on a commit to change the comparison range

Base

develop

d0a3601

-41.61%

add an in-place compress path using only safe code Using the FSST library currently requires you to make two allocations 1. Allocate a buffer to compress_into 2. Allocate a larger packed buffer We shouldn't take a &mut Vec<u8> directly, since Vec isn't splittable. We should instead be taking &mut [MaybeUninit<u8>], which can be backed by Vec, Bytes, Buffer<u8> or whatever other memory allocation we happen to get our hands on. This PR adds a new safe compression pathway that exposes `compress_into_uninit` and implements the hot loop using only safe code to boot. Signed-off-by: Andrew Duffy <andrew@a10y.dev>

feec50c

2 months ago

by a10y

+38.18%

elide bounds check (maybe?) Signed-off-by: Andrew Duffy <andrew@a10y.dev>

01ddd72

2 months ago

by a10y

-13.48%

maybe now elide bounds chekc? Signed-off-by: Andrew Duffy <andrew@a10y.dev>

7489f0d

2 months ago

by a10y

fix typos Signed-off-by: Andrew Duffy <andrew@a10y.dev>

5f10dd4

2 months ago

by a10y

Home Terms Privacy Docs