Avatar for the messense user
messense
jieba-rs
BlogDocsChangelog

feat: add `byte_start`/`byte_end` positions to `Token` and `Tag`

#143Merged
Comparing
add-byte-positions
(
e3d61a1
) with
main
(
afd80b3
)
CodSpeed Performance Gauge
-3%
Untouched
12

Benchmarks

12 total
multi_thread
jieba/benches/jieba_benchmark.rs::benches::criterion_benchmark::multithreaded
CodSpeed Performance Gauge
+3%
22.6 ms22 ms
default_mode
jieba/benches/jieba_benchmark.rs::benches::criterion_benchmark::tokenize
CodSpeed Performance Gauge
+2%
53 µs51.9 µs
new
jieba/benches/jieba_benchmark.rs::benches::criterion_benchmark::jieba
CodSpeed Performance Gauge
0%
390.5 ms390.7 ms
cut_for_search
jieba/benches/jieba_benchmark.rs::benches::criterion_benchmark::cut
CodSpeed Performance Gauge
-3%
57.9 µs59.5 µs
textrank
jieba/benches/jieba_benchmark.rs::benches::criterion_benchmark::keywords
CodSpeed Performance Gauge
-3%
74.6 µs76.8 µs
with_hmm
jieba/benches/jieba_benchmark.rs::benches::criterion_benchmark::cut
CodSpeed Performance Gauge
-3%
50.4 µs51.9 µs
single_thread
jieba/benches/jieba_benchmark.rs::benches::criterion_benchmark::multithreaded
CodSpeed Performance Gauge
-3%
21.2 ms22 ms
tag
jieba/benches/jieba_benchmark.rs::benches::criterion_benchmark::jieba
CodSpeed Performance Gauge
-4%
52.9 µs55 µs
no_hmm
jieba/benches/jieba_benchmark.rs::benches::criterion_benchmark::cut
CodSpeed Performance Gauge
-4%
36.3 µs37.8 µs
search_mode
jieba/benches/jieba_benchmark.rs::benches::criterion_benchmark::tokenize
CodSpeed Performance Gauge
-5%
56.4 µs59.6 µs
tfidf
jieba/benches/jieba_benchmark.rs::benches::criterion_benchmark::keywords
CodSpeed Performance Gauge
-6%
53.5 µs56.8 µs
cut_all
jieba/benches/jieba_benchmark.rs::benches::criterion_benchmark::cut
CodSpeed Performance Gauge
-6%
32.2 µs34.4 µs

Commits

Click on a commit to change the comparison range
Base
main
afd80b3
-10.88%
feat: add `byte_start`/`byte_end` positions to `Token` and `Tag`, change `cut*` to return `Vec<Token>`
4d09516
14 days ago
by messense
+0.08%
Use expect-test for snapshot testing
cdaf009
14 days ago
by messense
+8.07%
Eliminate `byte_to_unicode_table` allocation in `cut_all` path
e3d61a1
14 days ago
by messense
© 2026 CodSpeed Technology
Home Terms Privacy Docs