yk/add-tracing-to-bench-compare - Branch - paradigmxyz/reth

feat(bench-compare): add OTLP tracing support for baseline/feature comparison

#19626Merged

Comparing

yk/add-tracing-to-bench-compare

(

2d86f90

) with

main

(

9bc2bf2

)

-25%

Regressions: 1

Untouched: 80

Benchmarks

Failed

remove_leaf[1000]

crates/trie/sparse/benches/update.rs::benches::remove_leaf

Regression

-25%

199.2 µs264.3 µs

Passed

size 100000 | updated 0.1% | depth 2

crates/trie/sparse/benches/rlp_node.rs::rlp_node::update_rlp_node_level::update rlp node level

+3%

3.5 ms3.4 ms

size 100000 | updated 0.1% | depth 0

crates/trie/sparse/benches/rlp_node.rs::rlp_node::update_rlp_node_level::update rlp node level

+3%

3.7 ms3.6 ms

size 100000 | updated 0.1% | depth 1

crates/trie/sparse/benches/rlp_node.rs::rlp_node::update_rlp_node_level::update rlp node level

+3%

3.7 ms3.6 ms

size 100000 | updated 0.1% | depth 5

crates/trie/sparse/benches/rlp_node.rs::rlp_node::update_rlp_node_level::update rlp node level

+1%

1.1 ms1.1 ms

sparse trie[1000]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves::calculate root from leaves

+1%

5.7 ms5.7 ms

hash builder[init size 10000 | update size 100 | num updates 1]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

+1%

8.7 ms8.6 ms

size 100000 | updated 1% | depth 4

crates/trie/sparse/benches/rlp_node.rs::rlp_node::update_rlp_node_level::update rlp node level

+1%

11.4 ms11.3 ms

prefix set | size: 10 | `BTreeSet` with `BTreeSet:range` lookup

crates/trie/common/benches/prefix_set.rs::prefix_set::prefix_set_lookups::Prefix Set Lookups

+1%

5 µs5 µs

hash builder[init size 1000 | update size 100 | num updates 3]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

7.3 ms7.3 ms

sparse trie[init size 10000 | update size 1000 | num updates 3]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

33.9 ms33.8 ms

prefix set | size: 100 | `BTreeSet` with `BTreeSet:range` lookup

crates/trie/common/benches/prefix_set.rs::prefix_set::prefix_set_lookups::Prefix Set Lookups

40.4 µs40.4 µs

validate_blob | num blobs: 6 | ValidateBlob

crates/primitives/benches/validate_blob_tx.rs::validate_blob::blob_validation::Blob Transaction KZG validation

155 µs154.6 µs

prefix set | size: 1000 | `Vec` with binary search lookup

crates/trie/common/benches/prefix_set.rs::prefix_set::prefix_set_lookups::Prefix Set Lookups

205.7 µs205.4 µs

sparse trie[5000]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves::calculate root from leaves

28.4 ms28.4 ms

hash builder[init size 10000 | update size 1000 | num updates 1]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

25.5 ms25.4 ms

sparse trie[init size 1000 | update size 1000 | num updates 10]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

49.6 ms49.6 ms

hash builder[1000]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves::calculate root from leaves

4.5 ms4.5 ms

parallel hashing[1000]

crates/trie/trie/benches/hash_post_state.rs::post_state::hash_post_state::Hash Post State

2.6 s2.6 s

parallel hashing[100]

crates/trie/trie/benches/hash_post_state.rs::post_state::hash_post_state::Hash Post State

259 ms258.8 ms

receipts root | size: 10 | HashBuilder

crates/trie/trie/benches/trie_root.rs::benches::trie_root_benchmark::Receipts root calculation

112 µs111.9 µs

validate_blob | num blobs: 1 | ValidateBlob

crates/primitives/benches/validate_blob_tx.rs::validate_blob::blob_validation::Blob Transaction KZG validation

149.9 µs149.7 µs

sequence hashing[100]

crates/trie/trie/benches/hash_post_state.rs::post_state::hash_post_state::Hash Post State

258.9 ms258.7 ms

hash builder[init size 1000 | update size 100 | num updates 5]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

12.1 ms12.1 ms

validate_blob | num blobs: 3 | ValidateBlob

crates/primitives/benches/validate_blob_tx.rs::validate_blob::blob_validation::Blob Transaction KZG validation

150.9 µs150.8 µs

size 100000 | updated 0.1% | depth 3

crates/trie/sparse/benches/rlp_node.rs::rlp_node::update_rlp_node_level::update rlp node level

2.5 ms2.5 ms

validate_blob | num blobs: 2 | ValidateBlob

crates/primitives/benches/validate_blob_tx.rs::validate_blob::blob_validation::Blob Transaction KZG validation

149.6 µs149.6 µs

prefix set | size: 1000 | `Vec` with custom cursor lookup

crates/trie/common/benches/prefix_set.rs::prefix_set::prefix_set_lookups::Prefix Set Lookups

146.2 µs146.1 µs

prefix set | size: 1000 | `BTreeSet` with `BTreeSet:range` lookup

crates/trie/common/benches/prefix_set.rs::prefix_set::prefix_set_lookups::Prefix Set Lookups

472.9 µs472.8 µs

sparse trie[init size 1000 | update size 1000 | num updates 3]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

15 ms15 ms

hash builder[init size 1000 | update size 1000 | num updates 10]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

56.6 ms56.5 ms

prefix set | size: 1000 | `BTreeSet` with `Iterator:any` lookup

crates/trie/common/benches/prefix_set.rs::prefix_set::prefix_set_lookups::Prefix Set Lookups

14.9 ms14.9 ms

recover ECDSA

crates/primitives/benches/recover_ecdsa_crit.rs::benches::criterion_benchmark

206.7 µs206.7 µs

sparse trie[init size 10000 | update size 100 | num updates 1]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

2.3 ms2.3 ms

validate_blob | num blobs: 4 | ValidateBlob

crates/primitives/benches/validate_blob_tx.rs::validate_blob::blob_validation::Blob Transaction KZG validation

151.9 µs151.9 µs

ordered_trie_root

crates/trie/trie/benches/trie_root.rs::benches::trie_root_benchmark::Receipts root calculation::receipts root | size: 100 | triehash

1.2 ms1.2 ms

sparse trie[init size 1000 | update size 100 | num updates 1]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

1.1 ms1.1 ms

sparse trie[init size 10000 | update size 1000 | num updates 10]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

111.4 ms111.4 ms

ordered_trie_root

crates/trie/trie/benches/trie_root.rs::benches::trie_root_benchmark::Receipts root calculation::receipts root | size: 1000 | triehash

11.8 ms11.8 ms

receipts root | size: 1000 | HashBuilder

crates/trie/trie/benches/trie_root.rs::benches::trie_root_benchmark::Receipts root calculation

9.1 ms9.1 ms

prefix set | size: 10 | `Vec` with custom cursor lookup

crates/trie/common/benches/prefix_set.rs::prefix_set::prefix_set_lookups::Prefix Set Lookups

3.2 µs3.2 µs

prefix set | size: 100 | `BTreeSet` with `Iterator:any` lookup

crates/trie/common/benches/prefix_set.rs::prefix_set::prefix_set_lookups::Prefix Set Lookups

149.3 µs149.3 µs

sequence hashing[1000]

crates/trie/trie/benches/hash_post_state.rs::post_state::hash_post_state::Hash Post State

2.6 s2.6 s

sparse trie[init size 10000 | update size 100 | num updates 10]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

22 ms22 ms

hash builder[init size 1000 | update size 1000 | num updates 3]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

17 ms17 ms

sparse trie[init size 1000 | update size 1000 | num updates 5]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

24.9 ms24.9 ms

validate_blob | num blobs: 5 | ValidateBlob

crates/primitives/benches/validate_blob_tx.rs::validate_blob::blob_validation::Blob Transaction KZG validation

153.5 µs153.5 µs

ordered_trie_root

crates/trie/trie/benches/trie_root.rs::benches::trie_root_benchmark::Receipts root calculation::receipts root | size: 10 | triehash

131.7 µs131.7 µs

sparse trie[init size 10000 | update size 1000 | num updates 5]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

56 ms56 ms

prefix set | size: 10 | `Vec` with binary search lookup

crates/trie/common/benches/prefix_set.rs::prefix_set::prefix_set_lookups::Prefix Set Lookups

3.8 µs3.8 µs

hash builder[init size 1000 | update size 1000 | num updates 5]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

28.4 ms28.4 ms

receipts root | size: 100 | HashBuilder

crates/trie/trie/benches/trie_root.rs::benches::trie_root_benchmark::Receipts root calculation

937.5 µs937.9 µs

sparse trie[init size 10000 | update size 100 | num updates 3]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

6.7 ms6.7 ms

sparse trie[init size 10000 | update size 100 | num updates 5]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

11.1 ms11.1 ms

hash builder[5000]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves::calculate root from leaves

22.3 ms22.3 ms

hash builder[init size 10000 | update size 100 | num updates 5]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

40.5 ms40.5 ms

sparse trie[init size 1000 | update size 100 | num updates 10]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

10.8 ms10.8 ms

size 100000 | updated 0.1% | depth 4

crates/trie/sparse/benches/rlp_node.rs::rlp_node::update_rlp_node_level::update rlp node level

1.4 ms1.4 ms

sparse trie[init size 1000 | update size 100 | num updates 5]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

5.4 ms5.4 ms

hash builder[init size 10000 | update size 1000 | num updates 5]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

124.2 ms124.4 ms

size 100000 | updated 1% | depth 2

crates/trie/sparse/benches/rlp_node.rs::rlp_node::update_rlp_node_level::update rlp node level

24.7 ms24.8 ms

sparse trie[init size 1000 | update size 1000 | num updates 1]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

5.1 ms5.1 ms

size 100000 | updated 1% | depth 1

crates/trie/sparse/benches/rlp_node.rs::rlp_node::update_rlp_node_level::update rlp node level

24.9 ms24.9 ms

size 100000 | updated 1% | depth 0

crates/trie/sparse/benches/rlp_node.rs::rlp_node::update_rlp_node_level::update rlp node level

24.9 ms24.9 ms

hash builder[init size 10000 | update size 1000 | num updates 10]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

246.9 ms247.2 ms

hash builder[init size 10000 | update size 100 | num updates 3]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

24.9 ms24.9 ms

prefix set | size: 100 | `Vec` with binary search lookup

crates/trie/common/benches/prefix_set.rs::prefix_set::prefix_set_lookups::Prefix Set Lookups

19.4 µs19.4 µs

sparse trie[init size 1000 | update size 100 | num updates 3]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

3.3 ms3.3 ms

prefix set | size: 100 | `Vec` with custom cursor lookup

crates/trie/common/benches/prefix_set.rs::prefix_set::prefix_set_lookups::Prefix Set Lookups

16.7 µs16.7 µs

hash builder[init size 10000 | update size 1000 | num updates 3]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

75.1 ms75.2 ms

hash builder[init size 1000 | update size 100 | num updates 10]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

24 ms24 ms

sparse trie[init size 10000 | update size 1000 | num updates 1]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

11.6 ms11.7 ms

hash builder[init size 10000 | update size 100 | num updates 10]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

79.6 ms79.8 ms

hash builder[init size 1000 | update size 1000 | num updates 1]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

5.7 ms5.7 ms

size 100000 | updated 1% | depth 3

crates/trie/sparse/benches/rlp_node.rs::rlp_node::update_rlp_node_level::update rlp node level

21.8 ms21.9 ms

hash builder[init size 1000 | update size 100 | num updates 1]

crates/trie/sparse/benches/root.rs::root::calculate_root_from_leaves_repeated::calculate root from leaves repeated

-1%

2.4 ms2.4 ms

size 100000 | updated 1% | depth 5

crates/trie/sparse/benches/rlp_node.rs::rlp_node::update_rlp_node_level::update rlp node level

-1%

8.2 ms8.2 ms

prefix set | size: 10 | `BTreeSet` with `Iterator:any` lookup

crates/trie/common/benches/prefix_set.rs::prefix_set::prefix_set_lookups::Prefix Set Lookups

-2%

3.8 µs3.9 µs

update_leaf[1000]

crates/trie/sparse/benches/update.rs::benches::update_leaf

-3%

110.2 µs113.2 µs

remove_leaf[5000]

crates/trie/sparse/benches/update.rs::benches::remove_leaf

-4%

1.1 ms1.2 ms

update_leaf[5000]

crates/trie/sparse/benches/update.rs::benches::update_leaf

-7%

164.6 µs177.8 µs

Commits

Click on a commit to change the comparison range

Base

main

9bc2bf2

+0.09%

refactor: update tracing service name handling in NodeManager - Remove CLI flag for --tracing-otlp.service-name and set service name via OTEL_SERVICE_NAME environment variable. - Adjust documentation to reflect the new method of service name differentiation for tracing in Jaeger. - Ensure compatibility with existing tracing setup while simplifying the argument handling.

d248f39

4 hours ago

by yongkangc

+0.02%

revert

e5e6433

4 hours ago

by yongkangc

-26.19%

fix: add backticks to OTEL_SERVICE_NAME and long flag to service-name arg - Add backticks around OTEL_SERVICE_NAME in doc comment to fix clippy::doc-markdown - Add long flag 'tracing-otlp.service-name' to prevent clap from treating it as positional arg - Add help_heading for consistency with other tracing args

7b29d33

2 hours ago

by yongkangc

+0.65%

refactor: use CLI arg instead of env var for OTLP service name Address review feedback from @shekhirin to use --tracing-otlp.service-name CLI argument instead of OTEL_SERVICE_NAME environment variable. This approach provides: - Better consistency (all tracing config via CLI args) - Improved visibility in process listings - More explicit configuration Changes: - Add --tracing-otlp.service-name=reth-{ref_type} to CLI args - Remove OTEL_SERVICE_NAME env var setting - Keep OTEL_BLRP_MAX_QUEUE_SIZE env var (still needed)

c8abe65

2 hours ago

by yongkangc

+0.8%

revert: use OTEL_SERVICE_NAME env var instead of CLI arg Reverting c8abe6586 which attempted to use --tracing-otlp.service-name CLI flag. The CLI flag approach doesn't work because reth-bench-compare builds and runs reth binaries from arbitrary git refs (baseline-ref and feature-ref), which may not have the --tracing-otlp.service-name flag if they're older commits. Example failure: error: unexpected argument '--tracing-otlp.service-name' found The environment variable approach is correct here because: - OTEL_SERVICE_NAME is a standard OpenTelemetry variable - Works with any reth version that has OTLP support - Does not depend on the binary having the CLI flag This allows benchmarking between any two git refs, including older commits that predate the --tracing-otlp.service-name flag addition.

2d86f90

2 hours ago

by yongkangc

Home Terms Privacy Docs