Avatar for the armgilles user
armgilles
vcub_keeper
BlogDocsChangelog

Pandas VS Polars benchmark PR (don't merge)

#179
Comparing
dev_polars
(
0139174
) with
dev_pandas
(
62da224
)
CodSpeed Performance Gauge
+48%
Improvements
13
Regressions
0
Untouched
0
New
0
Dropped
0
Ignored
0

Benchmarks

Improved

test_benchmark_get_consecutive_no_transactions_out_big
tests/benchmarks/test_bench_transf_features.py::test_benchmark_get_consecutive_no_transactions_out_big
CodSpeed Performance Gauge
×10
609.7 ms
61.2 ms
test_benchmark_get_consecutive_no_transactions_out
tests/benchmarks/test_bench_transf_features.py::test_benchmark_get_consecutive_no_transactions_out
CodSpeed Performance Gauge
×9.6
19.5 ms
2 ms
test_benchmark_transf_json_to_df
tests/benchmarks/test_bench_transf_json_to_df.py::test_benchmark_transf_json_to_df
CodSpeed Performance Gauge
×6.6
159.4 ms
24.1 ms
test_benchmark_get_transaction_all
tests/benchmarks/test_bench_transf_features.py::test_benchmark_get_transaction_all
CodSpeed Performance Gauge
×4.8
7.5 ms
1.6 ms
test_benchmark_pipepline_transform
tests/benchmarks/test_bench_transf_json_to_df.py::test_benchmark_pipepline_transform
CodSpeed Performance Gauge
×4.8
834.6 ms
175 ms
test_benchmark_get_transaction_in
tests/benchmarks/test_bench_transf_features.py::test_benchmark_get_transaction_in
CodSpeed Performance Gauge
×4.7
8.7 ms
1.9 ms
test_benchmark_get_transaction_out
tests/benchmarks/test_bench_transf_features.py::test_benchmark_get_transaction_out
CodSpeed Performance Gauge
×4.5
8.5 ms
1.9 ms
test_benchmark_pipepline_transform_big
tests/benchmarks/test_bench_transf_json_to_df.py::test_benchmark_pipepline_transform_big
CodSpeed Performance Gauge
×4.2
8.4 s
2 s
test_benchmark_get_transaction_all_big
tests/benchmarks/test_bench_transf_features.py::test_benchmark_get_transaction_all_big
CodSpeed Performance Gauge
×3.7
192.6 ms
52 ms
test_benchmark_get_transaction_in_big
tests/benchmarks/test_bench_transf_features.py::test_benchmark_get_transaction_in_big
CodSpeed Performance Gauge
×3.5
186.2 ms
53.6 ms
test_benchmark_get_transaction_out_big
tests/benchmarks/test_bench_transf_features.py::test_benchmark_get_transaction_out_big
CodSpeed Performance Gauge
×3.4
176.9 ms
52.5 ms
test_benchmark_process_data_cluster
tests/benchmarks/test_bench_transf_features.py::test_benchmark_process_data_cluster
CodSpeed Performance Gauge
×3.1
12.9 ms
4.1 ms
test_benchmark_process_data_cluster_big
tests/benchmarks/test_bench_transf_features.py::test_benchmark_process_data_cluster_big
CodSpeed Performance Gauge
+48%
356.7 ms
240.8 ms

Commits

Click on a commit to change the comparison range
Base
dev_pandas
62da224
+0.08%
dev version 1.2.2a (#114) * Update for front (#111) * Fix install in front and got to be in PROD Signed-off-by: Armand <arm.gilles@gmail.com> * Just to check in site-package on dir below ROOT_DIR Signed-off-by: Armand <arm.gilles@gmail.com> * Update Ruff version in pre commit Signed-off-by: Armand <arm.gilles@gmail.com> * Change check for prod with new structure and Front Signed-off-by: Armand <arm.gilles@gmail.com> * Remove non usefull reset_index in oslandia API data crunsh Signed-off-by: Armand <arm.gilles@gmail.com> * Format Signed-off-by: Armand <arm.gilles@gmail.com> * Add matplotlib as dep (visualisation.py) Signed-off-by: Armand <arm.gilles@gmail.com> * Add seaborn as dep for deploy Signed-off-by: Armand <arm.gilles@gmail.com> * Try to fix error install ci watcher 'error: Your local changes to the following files would be overwritten by checkout' Signed-off-by: Armand <arm.gilles@gmail.com> * minor update on notebook Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com> * Bump to version 1.2.2a (#112) Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com>
fb7abd0
11 months ago
by armgilles
+0.85%
Polars read activity data (#119) * Update for front (#111) * Fix install in front and got to be in PROD Signed-off-by: Armand <arm.gilles@gmail.com> * Just to check in site-package on dir below ROOT_DIR Signed-off-by: Armand <arm.gilles@gmail.com> * Update Ruff version in pre commit Signed-off-by: Armand <arm.gilles@gmail.com> * Change check for prod with new structure and Front Signed-off-by: Armand <arm.gilles@gmail.com> * Remove non usefull reset_index in oslandia API data crunsh Signed-off-by: Armand <arm.gilles@gmail.com> * Format Signed-off-by: Armand <arm.gilles@gmail.com> * Add matplotlib as dep (visualisation.py) Signed-off-by: Armand <arm.gilles@gmail.com> * Add seaborn as dep for deploy Signed-off-by: Armand <arm.gilles@gmail.com> * Try to fix error install ci watcher 'error: Your local changes to the following files would be overwritten by checkout' Signed-off-by: Armand <arm.gilles@gmail.com> * minor update on notebook Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com> * Bump to version 1.2.2a (#112) Signed-off-by: Armand <arm.gilles@gmail.com> * Migre read_activity_vcub into Polars #117 Signed-off-by: Armand <arm.gilles@gmail.com> * Add Polars Pyarrow dep in package Signed-off-by: Armand <arm.gilles@gmail.com> * Improve migration #117 Signed-off-by: Armand <arm.gilles@gmail.com> * Update bench test #117 Signed-off-by: Armand <arm.gilles@gmail.com> * Fix pandas output in Bench tests #117 Signed-off-by: Armand <arm.gilles@gmail.com> * Typo Signed-off-by: Armand <arm.gilles@gmail.com> * forget to retour output with pandas... #117 Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com>
bcb86ef
11 months ago
by armgilles
×4.6
Polars get transaction out (#120) * Update for front (#111) * Fix install in front and got to be in PROD Signed-off-by: Armand <arm.gilles@gmail.com> * Just to check in site-package on dir below ROOT_DIR Signed-off-by: Armand <arm.gilles@gmail.com> * Update Ruff version in pre commit Signed-off-by: Armand <arm.gilles@gmail.com> * Change check for prod with new structure and Front Signed-off-by: Armand <arm.gilles@gmail.com> * Remove non usefull reset_index in oslandia API data crunsh Signed-off-by: Armand <arm.gilles@gmail.com> * Format Signed-off-by: Armand <arm.gilles@gmail.com> * Add matplotlib as dep (visualisation.py) Signed-off-by: Armand <arm.gilles@gmail.com> * Add seaborn as dep for deploy Signed-off-by: Armand <arm.gilles@gmail.com> * Try to fix error install ci watcher 'error: Your local changes to the following files would be overwritten by checkout' Signed-off-by: Armand <arm.gilles@gmail.com> * minor update on notebook Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com> * Bump to version 1.2.2a (#112) Signed-off-by: Armand <arm.gilles@gmail.com> * Add Polars dependencies to the project Signed-off-by: Armand <arm.gilles@gmail.com> * Adding pyarrow as dep for Polars Signed-off-by: Armand <arm.gilles@gmail.com> * Migrate get_transactions_out into Polars #116 Signed-off-by: Armand <arm.gilles@gmail.com> * Explicite return with pandas use case #116 Signed-off-by: Armand <arm.gilles@gmail.com> * Update bench test #116 Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com>
26dcd2a
11 months ago
by armgilles
+2.51%
Polars transactions in (#123) * Migrate transactions_in in Polars #121 Signed-off-by: Armand <arm.gilles@gmail.com> * Update test #121 Signed-off-by: Armand <arm.gilles@gmail.com> * Update read_activity_vcub to fix mistake output_type default and type Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com>
8270a28
11 months ago
by armgilles
+2.58%
Polars get transactions all (#125) * Migrate transactions_in in Polars #121 Signed-off-by: Armand <arm.gilles@gmail.com> * Update test #121 Signed-off-by: Armand <arm.gilles@gmail.com> * Update read_activity_vcub to fix mistake output_type default and type Signed-off-by: Armand <arm.gilles@gmail.com> * Migration transactions_all into Polars #124 Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com>
c440832
11 months ago
by armgilles
×2.5
Polars transform json api bdx station data to df (#128) * Migrate transactions_in in Polars #121 Signed-off-by: Armand <arm.gilles@gmail.com> * Update test #121 Signed-off-by: Armand <arm.gilles@gmail.com> * Update read_activity_vcub to fix mistake output_type default and type Signed-off-by: Armand <arm.gilles@gmail.com> * Migrate transform_json_api_bdx_station_data_to_df and oslandia func to polars #126 Signed-off-by: Armand <arm.gilles@gmail.com> * Update unit test & bench + data test for #126 Signed-off-by: Armand <arm.gilles@gmail.com> * Update test with #126 Signed-off-by: Armand <arm.gilles@gmail.com> * Update test with #126 Signed-off-by: Armand <arm.gilles@gmail.com> * Update test with #126 Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com>
efccc3e
11 months ago
by armgilles
Polars get encoding time (#131) * Migrate transactions_in in Polars #121 Signed-off-by: Armand <arm.gilles@gmail.com> * Update test #121 Signed-off-by: Armand <arm.gilles@gmail.com> * Update read_activity_vcub to fix mistake output_type default and type Signed-off-by: Armand <arm.gilles@gmail.com> * Working on #130 Signed-off-by: Armand <arm.gilles@gmail.com> * try to remove numpy and using math to have pi #130 Signed-off-by: Armand <arm.gilles@gmail.com> * test improve bench #130 Signed-off-by: Armand <arm.gilles@gmail.com> * test improve bench lit #130 Signed-off-by: Armand <arm.gilles@gmail.com> * test improve bench lit2 #130 Signed-off-by: Armand <arm.gilles@gmail.com> * test improve bench lit as 1 #130 Signed-off-by: Armand <arm.gilles@gmail.com> * test improve bench lit as expr #130 Signed-off-by: Armand <arm.gilles@gmail.com> * test improve bench lit as expr with mul#130 Signed-off-by: Armand <arm.gilles@gmail.com> * test improve bench lit as expr with no mul#130 Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com>
4e81203
11 months ago
by armgilles
Polars bigger dataset bench (#135) * Update for front (#111) * Fix install in front and got to be in PROD Signed-off-by: Armand <arm.gilles@gmail.com> * Just to check in site-package on dir below ROOT_DIR Signed-off-by: Armand <arm.gilles@gmail.com> * Update Ruff version in pre commit Signed-off-by: Armand <arm.gilles@gmail.com> * Change check for prod with new structure and Front Signed-off-by: Armand <arm.gilles@gmail.com> * Remove non usefull reset_index in oslandia API data crunsh Signed-off-by: Armand <arm.gilles@gmail.com> * Format Signed-off-by: Armand <arm.gilles@gmail.com> * Add matplotlib as dep (visualisation.py) Signed-off-by: Armand <arm.gilles@gmail.com> * Add seaborn as dep for deploy Signed-off-by: Armand <arm.gilles@gmail.com> * Try to fix error install ci watcher 'error: Your local changes to the following files would be overwritten by checkout' Signed-off-by: Armand <arm.gilles@gmail.com> * minor update on notebook Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com> * Bump to version 1.2.2a (#112) Signed-off-by: Armand <arm.gilles@gmail.com> * dev_pandas version 1.2.2a (#113) * Update for front (#111) * Fix install in front and got to be in PROD Signed-off-by: Armand <arm.gilles@gmail.com> * Just to check in site-package on dir below ROOT_DIR Signed-off-by: Armand <arm.gilles@gmail.com> * Update Ruff version in pre commit Signed-off-by: Armand <arm.gilles@gmail.com> * Change check for prod with new structure and Front Signed-off-by: Armand <arm.gilles@gmail.com> * Remove non usefull reset_index in oslandia API data crunsh Signed-off-by: Armand <arm.gilles@gmail.com> * Format Signed-off-by: Armand <arm.gilles@gmail.com> * Add matplotlib as dep (visualisation.py) Signed-off-by: Armand <arm.gilles@gmail.com> * Add seaborn as dep for deploy Signed-off-by: Armand <arm.gilles@gmail.com> * Try to fix error install ci watcher 'error: Your local changes to the following files would be overwritten by checkout' Signed-off-by: Armand <arm.gilles@gmail.com> * minor update on notebook Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com> * Bump to version 1.2.2a (#112) Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com> * Migrate transactions_in in Polars #121 Signed-off-by: Armand <arm.gilles@gmail.com> * Update test #121 Signed-off-by: Armand <arm.gilles@gmail.com> * Update read_activity_vcub to fix mistake output_type default and type Signed-off-by: Armand <arm.gilles@gmail.com> * Increase bench dataset (#134) * Update for front (#111) * Fix install in front and got to be in PROD Signed-off-by: Armand <arm.gilles@gmail.com> * Just to check in site-package on dir below ROOT_DIR Signed-off-by: Armand <arm.gilles@gmail.com> * Update Ruff version in pre commit Signed-off-by: Armand <arm.gilles@gmail.com> * Change check for prod with new structure and Front Signed-off-by: Armand <arm.gilles@gmail.com> * Remove non usefull reset_index in oslandia API data crunsh Signed-off-by: Armand <arm.gilles@gmail.com> * Format Signed-off-by: Armand <arm.gilles@gmail.com> * Add matplotlib as dep (visualisation.py) Signed-off-by: Armand <arm.gilles@gmail.com> * Add seaborn as dep for deploy Signed-off-by: Armand <arm.gilles@gmail.com> * Try to fix error install ci watcher 'error: Your local changes to the following files would be overwritten by checkout' Signed-off-by: Armand <arm.gilles@gmail.com> * minor update on notebook Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com> * Bump to version 1.2.2a (#112) Signed-off-by: Armand <arm.gilles@gmail.com> * Add bench test with bigger dataset #133 Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com> * Increase dateset bench with polars #133 Signed-off-by: Armand <arm.gilles@gmail.com> * Update input ouput bigger fonction to polars #133 Signed-off-by: Armand <arm.gilles@gmail.com> * Update input ouput bigger fonction to polars #133 Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com>
03d9839
11 months ago
by armgilles
-11.81%
Polars get consecutive no transactions out (#132) * Update for front (#111) * Fix install in front and got to be in PROD Signed-off-by: Armand <arm.gilles@gmail.com> * Just to check in site-package on dir below ROOT_DIR Signed-off-by: Armand <arm.gilles@gmail.com> * Update Ruff version in pre commit Signed-off-by: Armand <arm.gilles@gmail.com> * Change check for prod with new structure and Front Signed-off-by: Armand <arm.gilles@gmail.com> * Remove non usefull reset_index in oslandia API data crunsh Signed-off-by: Armand <arm.gilles@gmail.com> * Format Signed-off-by: Armand <arm.gilles@gmail.com> * Add matplotlib as dep (visualisation.py) Signed-off-by: Armand <arm.gilles@gmail.com> * Add seaborn as dep for deploy Signed-off-by: Armand <arm.gilles@gmail.com> * Try to fix error install ci watcher 'error: Your local changes to the following files would be overwritten by checkout' Signed-off-by: Armand <arm.gilles@gmail.com> * minor update on notebook Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com> * Bump to version 1.2.2a (#112) Signed-off-by: Armand <arm.gilles@gmail.com> * dev_pandas version 1.2.2a (#113) * Update for front (#111) * Fix install in front and got to be in PROD Signed-off-by: Armand <arm.gilles@gmail.com> * Just to check in site-package on dir below ROOT_DIR Signed-off-by: Armand <arm.gilles@gmail.com> * Update Ruff version in pre commit Signed-off-by: Armand <arm.gilles@gmail.com> * Change check for prod with new structure and Front Signed-off-by: Armand <arm.gilles@gmail.com> * Remove non usefull reset_index in oslandia API data crunsh Signed-off-by: Armand <arm.gilles@gmail.com> * Format Signed-off-by: Armand <arm.gilles@gmail.com> * Add matplotlib as dep (visualisation.py) Signed-off-by: Armand <arm.gilles@gmail.com> * Add seaborn as dep for deploy Signed-off-by: Armand <arm.gilles@gmail.com> * Try to fix error install ci watcher 'error: Your local changes to the following files would be overwritten by checkout' Signed-off-by: Armand <arm.gilles@gmail.com> * minor update on notebook Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com> * Bump to version 1.2.2a (#112) Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com> * Migrate transactions_in in Polars #121 Signed-off-by: Armand <arm.gilles@gmail.com> * Update test #121 Signed-off-by: Armand <arm.gilles@gmail.com> * Update read_activity_vcub to fix mistake output_type default and type Signed-off-by: Armand <arm.gilles@gmail.com> * Update unit test for #129 Signed-off-by: Armand <arm.gilles@gmail.com> * still wip #129 Signed-off-by: Armand <arm.gilles@gmail.com> * Fonction still en Pandas, but CI in Polars #129 Signed-off-by: Armand <arm.gilles@gmail.com> * Add in test data station eq 0 #129 Signed-off-by: Armand <arm.gilles@gmail.com> * Try a implementation of Polars migration but Slow I guess # 129 Signed-off-by: Armand <arm.gilles@gmail.com> * Increase bench dataset (#134) * Update for front (#111) * Fix install in front and got to be in PROD Signed-off-by: Armand <arm.gilles@gmail.com> * Just to check in site-package on dir below ROOT_DIR Signed-off-by: Armand <arm.gilles@gmail.com> * Update Ruff version in pre commit Signed-off-by: Armand <arm.gilles@gmail.com> * Change check for prod with new structure and Front Signed-off-by: Armand <arm.gilles@gmail.com> * Remove non usefull reset_index in oslandia API data crunsh Signed-off-by: Armand <arm.gilles@gmail.com> * Format Signed-off-by: Armand <arm.gilles@gmail.com> * Add matplotlib as dep (visualisation.py) Signed-off-by: Armand <arm.gilles@gmail.com> * Add seaborn as dep for deploy Signed-off-by: Armand <arm.gilles@gmail.com> * Try to fix error install ci watcher 'error: Your local changes to the following files would be overwritten by checkout' Signed-off-by: Armand <arm.gilles@gmail.com> * minor update on notebook Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com> * Bump to version 1.2.2a (#112) Signed-off-by: Armand <arm.gilles@gmail.com> * Add bench test with bigger dataset #133 Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com> * fix value in get_consecutive_no_transactions_out unit test Signed-off-by: Armand <arm.gilles@gmail.com> * Try to improve perf on big dataset #129 Signed-off-by: Armand <arm.gilles@gmail.com> * try an optimize way #129 Signed-off-by: Armand <arm.gilles@gmail.com> * Cleaning for #129 Signed-off-by: Armand <arm.gilles@gmail.com> * cleaning notebook #129 Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com>
5841367
11 months ago
by armgilles
+39%
Polars read station profile (#138) * Update for front (#111) * Fix install in front and got to be in PROD Signed-off-by: Armand <arm.gilles@gmail.com> * Just to check in site-package on dir below ROOT_DIR Signed-off-by: Armand <arm.gilles@gmail.com> * Update Ruff version in pre commit Signed-off-by: Armand <arm.gilles@gmail.com> * Change check for prod with new structure and Front Signed-off-by: Armand <arm.gilles@gmail.com> * Remove non usefull reset_index in oslandia API data crunsh Signed-off-by: Armand <arm.gilles@gmail.com> * Format Signed-off-by: Armand <arm.gilles@gmail.com> * Add matplotlib as dep (visualisation.py) Signed-off-by: Armand <arm.gilles@gmail.com> * Add seaborn as dep for deploy Signed-off-by: Armand <arm.gilles@gmail.com> * Try to fix error install ci watcher 'error: Your local changes to the following files would be overwritten by checkout' Signed-off-by: Armand <arm.gilles@gmail.com> * minor update on notebook Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com> * Bump to version 1.2.2a (#112) Signed-off-by: Armand <arm.gilles@gmail.com> * dev_pandas version 1.2.2a (#113) * Update for front (#111) * Fix install in front and got to be in PROD Signed-off-by: Armand <arm.gilles@gmail.com> * Just to check in site-package on dir below ROOT_DIR Signed-off-by: Armand <arm.gilles@gmail.com> * Update Ruff version in pre commit Signed-off-by: Armand <arm.gilles@gmail.com> * Change check for prod with new structure and Front Signed-off-by: Armand <arm.gilles@gmail.com> * Remove non usefull reset_index in oslandia API data crunsh Signed-off-by: Armand <arm.gilles@gmail.com> * Format Signed-off-by: Armand <arm.gilles@gmail.com> * Add matplotlib as dep (visualisation.py) Signed-off-by: Armand <arm.gilles@gmail.com> * Add seaborn as dep for deploy Signed-off-by: Armand <arm.gilles@gmail.com> * Try to fix error install ci watcher 'error: Your local changes to the following files would be overwritten by checkout' Signed-off-by: Armand <arm.gilles@gmail.com> * minor update on notebook Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com> * Bump to version 1.2.2a (#112) Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com> * Migrate transactions_in in Polars #121 Signed-off-by: Armand <arm.gilles@gmail.com> * Update test #121 Signed-off-by: Armand <arm.gilles@gmail.com> * Update read_activity_vcub to fix mistake output_type default and type Signed-off-by: Armand <arm.gilles@gmail.com> * Increase bench dataset (#134) * Update for front (#111) * Fix install in front and got to be in PROD Signed-off-by: Armand <arm.gilles@gmail.com> * Just to check in site-package on dir below ROOT_DIR Signed-off-by: Armand <arm.gilles@gmail.com> * Update Ruff version in pre commit Signed-off-by: Armand <arm.gilles@gmail.com> * Change check for prod with new structure and Front Signed-off-by: Armand <arm.gilles@gmail.com> * Remove non usefull reset_index in oslandia API data crunsh Signed-off-by: Armand <arm.gilles@gmail.com> * Format Signed-off-by: Armand <arm.gilles@gmail.com> * Add matplotlib as dep (visualisation.py) Signed-off-by: Armand <arm.gilles@gmail.com> * Add seaborn as dep for deploy Signed-off-by: Armand <arm.gilles@gmail.com> * Try to fix error install ci watcher 'error: Your local changes to the following files would be overwritten by checkout' Signed-off-by: Armand <arm.gilles@gmail.com> * minor update on notebook Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com> * Bump to version 1.2.2a (#112) Signed-off-by: Armand <arm.gilles@gmail.com> * Add bench test with bigger dataset #133 Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com> * wip #136 Signed-off-by: Armand <arm.gilles@gmail.com> * comment read_meto, old fonction stay here just to know Signed-off-by: Armand <arm.gilles@gmail.com> * Migrate create_station_attribute & read_stations_attributes to polars with unit test #137 Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com>
e2aefaf
11 months ago
by armgilles
-32.52%
Merge branch 'dev_polars' of github.com:armgilles/vcub_keeper into dev_polars
cfc2f27
11 months ago
by armgilles
-2.3%
Polars read station profile (#140) * Update omit coverage with no visualisation code Signed-off-by: Armand <arm.gilles@gmail.com> * migrate filter_periode to polars and add unit test Signed-off-by: Armand <arm.gilles@gmail.com> * To pass CI test with previous commit Signed-off-by: Armand <arm.gilles@gmail.com> * Migrate create & read station profile #136 Signed-off-by: Armand <arm.gilles@gmail.com> * Update & cleaning notebook #136 Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com>
d24fba4
11 months ago
by armgilles
-3.58%
Polars read time serie activity (#142) * Migrate create_activity_time_series & create_activity_time_series into partial polars with parquet & lazyframe #141 Signed-off-by: Armand <arm.gilles@gmail.com> * Addapt some code with read_time_serie_activity in lazyframe (not fully polars) #141 Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com>
e8ba1eb
11 months ago
by armgilles
+6.48%
Polars read time serie activity (#143) * Migrate create_activity_time_series & create_activity_time_series into partial polars with parquet & lazyframe #141 Signed-off-by: Armand <arm.gilles@gmail.com> * Addapt some code with read_time_serie_activity in lazyframe (not fully polars) #141 Signed-off-by: Armand <arm.gilles@gmail.com> * Addapt some code with read_time_serie_activity in lazyframe (not fully polars) #141 Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com>
03fd370
11 months ago
by armgilles
+3.16%
Using custum breakpoint calcul, no more pandas cut and add unit test #144 (#145) Signed-off-by: Armand <arm.gilles@gmail.com>
f15c810
11 months ago
by armgilles
-0.81%
Polars bench pipeline (#157) * Bench pipeline transf (#155) * Update for front (#111) * Fix install in front and got to be in PROD Signed-off-by: Armand <arm.gilles@gmail.com> * Just to check in site-package on dir below ROOT_DIR Signed-off-by: Armand <arm.gilles@gmail.com> * Update Ruff version in pre commit Signed-off-by: Armand <arm.gilles@gmail.com> * Change check for prod with new structure and Front Signed-off-by: Armand <arm.gilles@gmail.com> * Remove non usefull reset_index in oslandia API data crunsh Signed-off-by: Armand <arm.gilles@gmail.com> * Format Signed-off-by: Armand <arm.gilles@gmail.com> * Add matplotlib as dep (visualisation.py) Signed-off-by: Armand <arm.gilles@gmail.com> * Add seaborn as dep for deploy Signed-off-by: Armand <arm.gilles@gmail.com> * Try to fix error install ci watcher 'error: Your local changes to the following files would be overwritten by checkout' Signed-off-by: Armand <arm.gilles@gmail.com> * minor update on notebook Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com> * Bump to version 1.2.2a (#112) Signed-off-by: Armand <arm.gilles@gmail.com> * Add benchmark for pipeline transf from json API data #154 Signed-off-by: Armand <arm.gilles@gmail.com> * Improve speed of creation of simulated data & fix big data creation setting #154 Signed-off-by: Armand <arm.gilles@gmail.com> * Optimise creation of simulated data and reduce volume test #154 Signed-off-by: Armand <arm.gilles@gmail.com> * Codspeed run forever, only on small test #154 Signed-off-by: Armand <arm.gilles@gmail.com> * Codspeed run forever, only on small test #154 Signed-off-by: Armand <arm.gilles@gmail.com> * Codspeed previous commit ok with no new test, check with one Signed-off-by: Armand <arm.gilles@gmail.com> * Reduce volume for #154 to run on codspeed Signed-off-by: Armand <arm.gilles@gmail.com> * Reduce volume for #154 to run on codspeed Signed-off-by: Armand <arm.gilles@gmail.com> * Reduce again volume for codspeed, reduce CI time Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com> * specify timezone to be as real data #154 Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com>
c247257
11 months ago
by armgilles
+47.66%
merging from dev_polars to have bench pipeline tests Signed-off-by: Armand <arm.gilles@gmail.com>
25bc431
11 months ago
by armgilles
-1.48%
Try lazy + expression fonction to check perf bench #146 (#147) * Try lazy + expression fonction to check perf bench #146 Signed-off-by: Armand <arm.gilles@gmail.com> * Update code with get_transactions_out expr #146 Signed-off-by: Armand <arm.gilles@gmail.com> * have to collect with lazy #146 Signed-off-by: Armand <arm.gilles@gmail.com> * Lazy read_activity_vcub #148 and update notebook transactions_out Signed-off-by: Armand <arm.gilles@gmail.com> * Update bench test with lazy vcub_keeper_py312 #148 Signed-off-by: Armand <arm.gilles@gmail.com> * update docstring #148 Signed-off-by: Armand <arm.gilles@gmail.com> * collect lazy df to be a fair bench #146 Signed-off-by: Armand <arm.gilles@gmail.com> * collect lazy df to be a fair bench #146 Signed-off-by: Armand <arm.gilles@gmail.com> * collect lazy df to be a fair bench #146 Signed-off-by: Armand <arm.gilles@gmail.com> * Update transactions_in to be lazy and expr fonction #149 Signed-off-by: Armand <arm.gilles@gmail.com> * Try to improve bench test on big result Signed-off-by: Armand <arm.gilles@gmail.com> * lazy and Expr function for transactions_all function #150 Signed-off-by: Armand <arm.gilles@gmail.com> * try to fix bad perf on big dataset lazy Signed-off-by: Armand <arm.gilles@gmail.com> * try to fix bad perf on big dataset lazy Signed-off-by: Armand <arm.gilles@gmail.com> * try to fix bad perf on big dataset lazy Signed-off-by: Armand <arm.gilles@gmail.com> * Lazy expr for get_consecutive_no_transactions_out #151 Signed-off-by: Armand <arm.gilles@gmail.com> * Lazy transform_json_api_bdx_station_data_to_df function #152 Signed-off-by: Armand <arm.gilles@gmail.com> * Encoding time in Expr function and process_data_cluster in lazy mode #153 Signed-off-by: Armand <arm.gilles@gmail.com> * add todo for ML with pandas Signed-off-by: Armand <arm.gilles@gmail.com> * Adapt code for pipeline bench lazy Signed-off-by: Armand <arm.gilles@gmail.com> * Try new lazy for pipeline bench Signed-off-by: Armand <arm.gilles@gmail.com> * Try new lazy for pipeline bench Signed-off-by: Armand <arm.gilles@gmail.com> * process data with with_columns style & lazy #161 Signed-off-by: Armand <arm.gilles@gmail.com> * forget previous commit Signed-off-by: Armand <arm.gilles@gmail.com> * have to collect this tests Signed-off-by: Armand <arm.gilles@gmail.com> * Small test bench are in eager mode, big in lazy mode to faire comparaison Signed-off-by: Armand <arm.gilles@gmail.com> * Small test bench are in eager mode, big in lazy mode to faire comparaison Signed-off-by: Armand <arm.gilles@gmail.com> * Update notebook with with_colums style for feature creation Signed-off-by: Armand <arm.gilles@gmail.com> * Using pipe style with lazy Signed-off-by: Armand <arm.gilles@gmail.com> * cleaning Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com>
15687d4
11 months ago
by armgilles
-49.29%
Update ML pipeple to Polars #162 (#163) Signed-off-by: Armand <arm.gilles@gmail.com>
47ef206
11 months ago
by armgilles
+48.81%
Fix new value status from API Bordeaux, improve mapping and test #165 (#166) Signed-off-by: Armand <arm.gilles@gmail.com>
614f2f4
11 months ago
by armgilles
-49.48%
Polars viz (#167) * Update important viz fucntion to polars #164 Signed-off-by: Armand <arm.gilles@gmail.com> * Upate notebook with viz, some pandas are still here but it's ok #164 Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com>
60afa5f
11 months ago
by armgilles
+2.61%
Change date type into Bdx pipeline (unset time zone) #168) (#169) Signed-off-by: Armand <arm.gilles@gmail.com>
3fee3a1
11 months ago
by armgilles
-0.24%
Fix available bikes consecutive feat (#171) * Fix bad condition for consecutive_no_transactions_out and available bike in station #170 Signed-off-by: Armand <arm.gilles@gmail.com> * Update test to test available_bike less or equal 2 is consecutive_no_transactions_out = 0 #170 Signed-off-by: Armand <arm.gilles@gmail.com> * Small update to give same type in test #170 Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com>
ab1fa8c
11 months ago
by armgilles
-2%
try to fix problem with old lfs notebook error #172 Signed-off-by: Armand <arm.gilles@gmail.com>
7378bf3
10 months ago
by armgilles
+3.32%
Bump pyproject and add dep to polars to read from cloud (#175) * Bump pyproject and add dep to polars to read from cloud Signed-off-by: Armand <arm.gilles@gmail.com> * Fix Deprecated warning with replace to replace_strict Signed-off-by: Armand <arm.gilles@gmail.com> * Fix Deprecated warning with check_dtype to check_dtypes Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com>
983311c
10 months ago
by armgilles
+1.12%
Add manifest for notebook (#176) * Adding MANIFEST to exclude notebook Signed-off-by: Armand <arm.gilles@gmail.com> * Exclude notebook and md files during install Signed-off-by: Armand <arm.gilles@gmail.com> * remove .gitattributes lfs files Signed-off-by: Armand <arm.gilles@gmail.com> * Add a blanck .gitattributes file Signed-off-by: Armand <arm.gilles@gmail.com> * Supprimer Git LFS et exclure les notebooks du suivi LFS Signed-off-by: Armand <arm.gilles@gmail.com> * Remove .gitattributes file Signed-off-by: Armand <arm.gilles@gmail.com> --------- Signed-off-by: Armand <arm.gilles@gmail.com>
b7eaeeb
10 months ago
by armgilles
-7.1%
Fix lfs (#177) * Corriger le suivi Git LFS pour les fichiers notebooks et .ipynb * Add again .gitattribute Signed-off-by: Armand <arm.gilles@gmail.com> * Fix: Track notebooks with Git LFS * Fix: Reindex notebooks for Git LFS --------- Signed-off-by: Armand <arm.gilles@gmail.com>
d08b5f9
10 months ago
by armgilles
+62.41%
Merge branch 'dev_polars' of https://github.com/armgilles/vcub_keeper into dev_polars
0139174
10 months ago
by armgilles
© 2025 CodSpeed Technology
Home Terms Privacy Docs