NAME
Bencher::Scenario::ComparingArrays - Modules that compare arrays
VERSION
This document describes version 0.002 of Bencher::Scenario::ComparingArrays (from Perl distribution Bencher-Scenario-ComparingArrays), released on 2019-03-24.
SYNOPSIS
To run benchmark with default option:
% bencher -m ComparingArraysTo run module startup overhead benchmark:
% bencher --module-startup -m ComparingArraysFor more options (dump scenario, list/include/exclude/add participants, list/include/exclude/add datasets, etc), see bencher or run bencher --help.
DESCRIPTION
Packaging a benchmark script as a Bencher scenario makes it convenient to include/exclude/add participants/datasets (either via CLI or Perl code), send the result to a central repository, among others . See Bencher and bencher (CLI) for more details.
BENCHMARKED MODULES
Version numbers shown below are the versions used when running the sample benchmark.
Array::Compare 3.0.2
Arrays::Same 0.002
Cpanel::JSON::XS 3.0239
Data::Cmp 0.006
Sereal::Encoder 4.005
Storable 2.62
BENCHMARK PARTICIPANTS
Data::Cmp::cmp_data (perl_code) [int, str]
Function call template:
Data::Cmp::cmp_data(<array1>, <array2>)Array::Compare (perl_code) [int, str]
Code template:
Array::Compare->new()->compare(<array1>, <array2>)Arrays::Same::arrays_same_i (perl_code) [int]
Function call template:
Arrays::Same::arrays_same_i(<array1>, <array2>)Arrays::Same::arrays_same_s (perl_code) [str]
Function call template:
Arrays::Same::arrays_same_s(<array1>, <array2>)num eq op loop (perl_code) [int]
Code template:
Bencher::Scenario::ComparingArrays::_loop(<array1>, <array2>)str eq op loop (perl_code) [str]
Code template:
Bencher::Scenario::ComparingArrays::_loopstr(<array1>, <array2>)Storable (perl_code) [int, str]
Code template:
Storable::freeze(<array1>) eq Storable::freeze(<array2>)Sereal::Encoder (perl_code) [int, str]
Code template:
Sereal::Encoder::encode_sereal(<array1>) eq Sereal::Encoder::encode_sereal(<array2>)Cpanel::JSON::XS (perl_code) [int, str]
Code template:
Cpanel::JSON::XS::encode_json(<array1>) eq Cpanel::JSON::XS::encode_json(<array2>)
BENCHMARK DATASETS
empty
10-int-same
10-str-same
100-int-same
100-str-same
1000-int-same
1000-str-same
SAMPLE BENCHMARK RESULTS
Run on: perl: v5.26.1, CPU: Intel(R) Core(TM) M-5Y71 CPU @ 1.20GHz (2 cores), OS: GNU/Linux LinuxMint version 18.3, OS kernel: Linux version 4.10.0-38-generic.
Benchmark with default options (bencher -m ComparingArrays):
#table1#
{dataset=>"10-int-same"}
+-----------------------------+----------+-----------+-----------+------------+---------+---------+
| participant | p_tags | rate (/s) | time (μs) | vs_slowest | errors | samples |
+-----------------------------+----------+-----------+-----------+------------+---------+---------+
| Array::Compare | int, str | 44800 | 22.3 | 1 | 6.7e-09 | 20 |
| Data::Cmp::cmp_data | int, str | 88400 | 11.3 | 1.97 | 1e-08 | 34 |
| Storable | int, str | 142000 | 7.03 | 3.18 | 2.8e-09 | 28 |
| num eq op loop | int | 520000 | 1.92 | 11.6 | 6.5e-10 | 33 |
| Sereal::Encoder | int, str | 599717 | 1.66745 | 13.3943 | 0 | 24 |
| Cpanel::JSON::XS | int, str | 670000 | 1.5 | 15 | 1.7e-09 | 20 |
| Arrays::Same::arrays_same_i | int | 1210000 | 0.824 | 27.1 | 4.5e-10 | 20 |
+-----------------------------+----------+-----------+-----------+------------+---------+---------+
#table2#
{dataset=>"10-str-same"}
+-----------------------------+----------+-----------+-----------+------------+---------+---------+
| participant | p_tags | rate (/s) | time (μs) | vs_slowest | errors | samples |
+-----------------------------+----------+-----------+-----------+------------+---------+---------+
| Array::Compare | int, str | 43000 | 23 | 1 | 5.3e-08 | 20 |
| Data::Cmp::cmp_data | int, str | 81100 | 12.3 | 1.89 | 3.3e-09 | 21 |
| Storable | int, str | 130605 | 7.65669 | 3.03704 | 0 | 20 |
| str eq op loop | str | 348000 | 2.87 | 8.1 | 8.3e-10 | 20 |
| Cpanel::JSON::XS | int, str | 420000 | 2.4 | 9.8 | 5.3e-09 | 32 |
| Sereal::Encoder | int, str | 430000 | 2.3 | 10 | 3.1e-09 | 23 |
| Arrays::Same::arrays_same_s | str | 590000 | 1.7 | 14 | 3.3e-09 | 20 |
+-----------------------------+----------+-----------+-----------+------------+---------+---------+
#table3#
{dataset=>"100-int-same"}
+-----------------------------+----------+-----------+-----------+------------+---------+---------+
| participant | p_tags | rate (/s) | time (μs) | vs_slowest | errors | samples |
+-----------------------------+----------+-----------+-----------+------------+---------+---------+
| Data::Cmp::cmp_data | int, str | 11500 | 87.1 | 1 | 2.7e-08 | 20 |
| Array::Compare | int, str | 15000 | 67 | 1.3 | 3.5e-07 | 20 |
| Storable | int, str | 63000 | 16 | 5.5 | 1.1e-07 | 20 |
| num eq op loop | int | 69000 | 15 | 6 | 1.1e-07 | 20 |
| Cpanel::JSON::XS | int, str | 90000 | 11 | 7.9 | 1.6e-08 | 23 |
| Sereal::Encoder | int, str | 127000 | 7.84 | 11.1 | 3.3e-09 | 20 |
| Arrays::Same::arrays_same_i | int | 180000 | 5.5 | 16 | 6.5e-09 | 21 |
+-----------------------------+----------+-----------+-----------+------------+---------+---------+
#table4#
{dataset=>"100-str-same"}
+-----------------------------+----------+-----------+-----------+------------+---------+---------+
| participant | p_tags | rate (/s) | time (μs) | vs_slowest | errors | samples |
+-----------------------------+----------+-----------+-----------+------------+---------+---------+
| Data::Cmp::cmp_data | int, str | 11000 | 94 | 1 | 1e-07 | 21 |
| Array::Compare | int, str | 16000 | 63 | 1.5 | 1.1e-07 | 20 |
| Storable | int, str | 37200 | 26.9 | 3.48 | 1.3e-08 | 20 |
| str eq op loop | str | 47175.7 | 21.1973 | 4.41521 | 0 | 20 |
| Cpanel::JSON::XS | int, str | 60000 | 17 | 5.6 | 2.7e-08 | 20 |
| Sereal::Encoder | int, str | 66000 | 15 | 6.2 | 4.9e-08 | 24 |
| Arrays::Same::arrays_same_s | str | 77000 | 13 | 7.2 | 2e-08 | 21 |
+-----------------------------+----------+-----------+-----------+------------+---------+---------+
#table5#
{dataset=>"1000-int-same"}
+-----------------------------+----------+-----------+-----------+------------+---------+---------+
| participant | p_tags | rate (/s) | time (μs) | vs_slowest | errors | samples |
+-----------------------------+----------+-----------+-----------+------------+---------+---------+
| Data::Cmp::cmp_data | int, str | 1200 | 870 | 1 | 2e-06 | 21 |
| Array::Compare | int, str | 2100 | 470 | 1.8 | 9.1e-07 | 20 |
| num eq op loop | int | 6900 | 150 | 6 | 2.1e-07 | 20 |
| Storable | int, str | 7400 | 140 | 6.4 | 1.6e-07 | 20 |
| Cpanel::JSON::XS | int, str | 10000 | 96 | 9 | 1.1e-07 | 20 |
| Sereal::Encoder | int, str | 15000 | 69 | 13 | 2.1e-07 | 20 |
| Arrays::Same::arrays_same_i | int | 16500 | 60.5 | 14.4 | 2.7e-08 | 20 |
+-----------------------------+----------+-----------+-----------+------------+---------+---------+
#table6#
{dataset=>"1000-str-same"}
+-----------------------------+----------+-----------+-----------+------------+---------+---------+
| participant | p_tags | rate (/s) | time (μs) | vs_slowest | errors | samples |
+-----------------------------+----------+-----------+-----------+------------+---------+---------+
| Data::Cmp::cmp_data | int, str | 1100 | 900 | 1 | 2.7e-06 | 22 |
| Array::Compare | int, str | 2180 | 458 | 1.97 | 2.7e-07 | 20 |
| Storable | int, str | 4660 | 215 | 4.21 | 1.8e-07 | 27 |
| str eq op loop | str | 4836.24 | 206.772 | 4.37298 | 0 | 20 |
| Cpanel::JSON::XS | int, str | 6539.15 | 152.925 | 5.91276 | 0 | 20 |
| Sereal::Encoder | int, str | 7100 | 140 | 6.4 | 2.5e-07 | 23 |
| Arrays::Same::arrays_same_s | str | 7800 | 130 | 7 | 2.1e-07 | 20 |
+-----------------------------+----------+-----------+-----------+------------+---------+---------+
#table7#
{dataset=>"empty"}
+-----------------------------+----------+-----------+-----------+------------+---------+---------+
| participant | p_tags | rate (/s) | time (μs) | vs_slowest | errors | samples |
+-----------------------------+----------+-----------+-----------+------------+---------+---------+
| Array::Compare | int, str | 57000 | 17 | 1 | 2.7e-08 | 20 |
| Storable | int, str | 173526 | 5.76282 | 3.02255 | 0 | 20 |
| Data::Cmp::cmp_data | int, str | 390000 | 2.6 | 6.8 | 3.3e-09 | 20 |
| Sereal::Encoder | int, str | 1130000 | 0.888 | 19.6 | 4.1e-10 | 21 |
| str eq op loop | str | 1700000 | 0.6 | 29 | 8.1e-10 | 21 |
| num eq op loop | int | 1800000 | 0.55 | 32 | 2.6e-09 | 21 |
| Cpanel::JSON::XS | int, str | 1800000 | 0.55 | 32 | 7.8e-10 | 24 |
| Arrays::Same::arrays_same_s | str | 4400000 | 0.23 | 76 | 4.2e-10 | 20 |
| Arrays::Same::arrays_same_i | int | 4800000 | 0.21 | 83 | 4.2e-10 | 20 |
+-----------------------------+----------+-----------+-----------+------------+---------+---------+Benchmark module startup overhead (bencher -m ComparingArrays --module-startup):
#table8#
+---------------------+-----------+------------------------+------------+---------+---------+
| participant | time (ms) | mod_overhead_time (ms) | vs_slowest | errors | samples |
+---------------------+-----------+------------------------+------------+---------+---------+
| Array::Compare | 55 | 49.1 | 1 | 9.8e-05 | 20 |
| Storable | 14 | 8.1 | 4 | 2.5e-05 | 20 |
| Sereal::Encoder | 12 | 6.1 | 4.5 | 2e-05 | 20 |
| Cpanel::JSON::XS | 11 | 5.1 | 4.9 | 1.8e-05 | 20 |
| Arrays::Same | 10 | 4.1 | 5.4 | 1.1e-05 | 24 |
| Data::Cmp | 9.5 | 3.6 | 5.8 | 2.3e-05 | 20 |
| perl -e1 (baseline) | 5.9 | 0 | 9.3 | 4.3e-05 | 20 |
+---------------------+-----------+------------------------+------------+---------+---------+To display as an interactive HTML table on a browser, you can add option --format html+datatables.
HOMEPAGE
Please visit the project's homepage at https://metacpan.org/release/Bencher-Scenario-ComparingArrays.
SOURCE
Source repository is at https://github.com/perlancar/perl-Bencher-Scenario-ComparingArrays.
BUGS
Please report any bugs or feature requests on the bugtracker website https://rt.cpan.org/Public/Dist/Display.html?Name=Bencher-Scenario-ComparingArrays
When submitting a bug or request, please include a test-file or a patch to an existing test-file that illustrates the bug or desired feature.
AUTHOR
perlancar <perlancar@cpan.org>
COPYRIGHT AND LICENSE
This software is copyright (c) 2019 by perlancar@cpan.org.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.