feat/mdbx-parallel-writes - Branch - paradigmxyz/reth

feat: mdbx experimentation

#21776

Comparing

feat/mdbx-parallel-writes

(

56001ef

) with

main

(

9a026ec

)

Untouched: 118

Skipped: 7

Benchmarks

125 total

collect_twice[100]

crates/trie/common/benches/hashed_state.rs::benches::bench_from_parallel_iterator::HashedPostState::from_par_iter

+5%

93.3 µs88.4 µs

V2[dataset_128/targets_512]

crates/trie/trie/benches/proof_v2.rs::proof_comparison::bench_proof_algos::Proof

+3%

822.4 µs801.6 µs

V2[dataset_128/targets_2048]

crates/trie/trie/benches/proof_v2.rs::proof_comparison::bench_proof_algos::Proof

+3%

988.9 µs964.7 µs

size 100000 | updated 0.1% | depth 3

crates/trie/sparse/benches/rlp_node.rs::rlp_node::update_rlp_node_level::update rlp node level

+2%

2.4 ms2.4 ms

size 100000 | updated 0.1% | depth 5

crates/trie/sparse/benches/rlp_node.rs::rlp_node::update_rlp_node_level::update rlp node level

+2%

1.1 ms1.1 ms

prefix set | size: 10 | `BTreeSet` with `BTreeSet:range` lookup

crates/trie/common/benches/prefix_set.rs::prefix_set::prefix_set_lookups::Prefix Set Lookups

+2%

5.4 µs5.3 µs

fold_reduce[100]

crates/trie/common/benches/hashed_state.rs::benches::bench_from_parallel_iterator::HashedPostState::from_par_iter

+1%

161.1 µs158.7 µs

size 100000 | updated 1% | depth 4

crates/trie/sparse/benches/rlp_node.rs::rlp_node::update_rlp_node_level::update rlp node level

+1%

11.3 ms11.2 ms

size 100000 | updated 0.1% | depth 4

crates/trie/sparse/benches/rlp_node.rs::rlp_node::update_rlp_node_level::update rlp node level

+1%

1.4 ms1.4 ms

size 100000 | updated 1% | depth 5

crates/trie/sparse/benches/rlp_node.rs::rlp_node::update_rlp_node_level::update rlp node level

+1%

8.4 ms8.3 ms

hash builder[init size 10000 | update size 100 | num updates 1]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

+1%

9.7 ms9.6 ms

size 100000 | updated 1% | depth 3

crates/trie/sparse/benches/rlp_node.rs::rlp_node::update_rlp_node_level::update rlp node level

+1%

21.3 ms21 ms

size 100000 | updated 1% | depth 1

crates/trie/sparse/benches/rlp_node.rs::rlp_node::update_rlp_node_level::update rlp node level

+1%

24.1 ms23.9 ms

size 100000 | updated 1% | depth 0

crates/trie/sparse/benches/rlp_node.rs::rlp_node::update_rlp_node_level::update rlp node level

+1%

24.1 ms23.9 ms

size 100000 | updated 1% | depth 2

crates/trie/sparse/benches/rlp_node.rs::rlp_node::update_rlp_node_level::update rlp node level

+1%

24 ms23.8 ms

prefix set | size: 10 | `BTreeSet` with `Iterator:any` lookup

crates/trie/common/benches/prefix_set.rs::prefix_set::prefix_set_lookups::Prefix Set Lookups

+1%

3.9 µs3.9 µs

hash builder[init size 1000 | update size 100 | num updates 5]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

+1%

12.6 ms12.5 ms

collect_twice[1000]

crates/trie/common/benches/hashed_state.rs::benches::bench_from_parallel_iterator::HashedPostState::from_par_iter

+1%

727.2 µs722.7 µs

hash builder[init size 1000 | update size 100 | num updates 3]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

7.6 ms7.6 ms

V2[dataset_1024/targets_64]

crates/trie/trie/benches/proof_v2.rs::proof_comparison::bench_proof_algos::Proof

10.5 ms10.4 ms

prefix set | size: 1000 | `Vec` with custom cursor lookup

crates/trie/common/benches/prefix_set.rs::prefix_set::prefix_set_lookups::Prefix Set Lookups

156.9 µs156.6 µs

hash builder[init size 10000 | update size 100 | num updates 5]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

44.6 ms44.5 ms

sparse trie[init size 10000 | update size 1000 | num updates 3]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

33 ms32.9 ms

hash builder[init size 1000 | update size 100 | num updates 10]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

25 ms25 ms

sparse trie[init size 1000 | update size 100 | num updates 1]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

1.1 ms1.1 ms

Commits

Click on a commit to change the comparison range

Base

main

9a026ec

+0.03%

feat(provider): create subtxns only for tables written in save_blocks

d75a738

23 hours ago

by joshieDo

+0.2%

chore: remove unused Duration import

33dec9a

6 hours ago

by joshieDo

-0.21%

feat(provider): parallelize write_state, write_hashed_state, write_trie_updates_sorted in edge mode

7e50bda

6 hours ago

by joshieDo

-0.03%

fix(provider): move enable_parallel_writes after insert_block_mdbx_only

2a05c81

6 hours ago

by joshieDo

+0.29%

fix(mdbx): handle NULL repnl in txn_refund for parallel subtxns

11e4bb8

4 hours ago

by joshieDo

-0.04%

debug: add more tree_search debug output

2e65411

4 hours ago

by joshieDo

-0.02%

fix: sync geo.first_unallocated across sibling subtxns

6321aa0

4 hours ago

by joshieDo

fix: add sibling sync to gc_alloc_ex for multi-page allocations

03cb74c

4 hours ago

by joshieDo

-0.15%

test: expand test_save_blocks_edge_mode_parallel with large data

66bae88

4 hours ago

by joshieDo

-0.18%

debug: add tracing after tree_search in parallel cursor path

59f578e

4 hours ago

by joshieDo

-0.19%

test: add stress tests for parallel subtxn edge mode

196fef5

4 hours ago

by joshieDo

+0.07%

fix: allocate retired_pages and repnl for parallel subtxns

db1f916

3 hours ago

by joshieDo

+0.05%

fix: add NULL checks for iov_base in decode path

b33ec2e

3 hours ago

by joshieDo

-0.01%

fix(libmdbx): fix DEBUG macro calls to use format args

909dd60

3 hours ago

by joshieDo

+0.09%

test(libmdbx): add write/read verification to DupSort tests

19a3f48

3 hours ago

by joshieDo

-0.14%

fix(libmdbx): allocate page_auxbuf for parallel subtxns

07ffd87

2 hours ago

by joshieDo

+0.42%

docs(libmdbx): document parallel subtxn changes and invariants

a078448

2 hours ago

by joshieDo

-0.05%

docs(libmdbx): document all parallel subtxn invariants

48ba06a

2 hours ago

by joshieDo

-0.15%

fix(libmdbx): merge loose pages from parallel subtxns to parent

adbf483

2 hours ago

by joshieDo

-0.06%

fix(libmdbx): merge unused subtxn_repnl pages back to parent on commit

71758ff

2 hours ago

by joshieDo

+0.21%

fix(libmdbx): fix page validity checks for parallel subtxns

7720e16

1 hour ago

by joshieDo

-0.05%

fix(libmdbx): parent must use subtxn_alloc_mutex when it has active subtxns

41a07e0

55 minutes ago

by joshieDo

+0.15%

fix(libmdbx): block parent writes while parallel subtxns are active

26c2866

47 minutes ago

by joshieDo

-0.01%

fix(libmdbx): fix pnl_check validations for parallel subtxns

56001ef

17 minutes ago

by joshieDo

Home Terms Privacy Docs