Kwker benchmark results: DataFrames and analytics (group-by, table sorts, ORDER BY ... LIMIT) ============================================================================================= Published on https://kwker.io/benchmarks/#frames and https://kwker.io/data/. Raw output of python -m kwker.bench --frames as recorded (kwker/_bench_frames.py; the same cases anyone can run on their own machine). Machine Intel Xeon (Cascade Lake class, AVX-512 + VNNI), cloud VM, 4 cores; production build of Kwker (AVX-512 engine) Date 2026-10-08 (earlier version of this file: 2026-10-01) Libraries the versions printed below, same thread count as Kwker Data 1,000,000 rows. Group-by: one int64 key of 1,000 values with sum, mean, min, max and count of a float column (plus the exact median), or two keys (an int64 of 50 values and a string of 100 values) with sum, mean and count. Table sorts: 5 columns, sorted by the named key column(s). ORDER BY g, f LIMIT k: g = int64 of 1,000 values, f = float64. Every Kwker result is checked against the library's own result first, under that library's null and NaN rules; a case whose results differ is reported and not timed. Protocol best of 3 per library and case; speed-up = library time / Kwker time. "polars + sort" = Polars group_by(..).agg(..).sort(by): groups in key order, as Kwker returns them. Group-by: 4 threads each (python -m kwker.bench --frames --threads 4); table sorts and ORDER BY ... LIMIT: 1 thread each (python -m kwker.bench --frames). Columns below: suite, library, case, the library's ms, Kwker's ms, speed-up. The table sorts and ORDER BY rows are the second 1-thread run of the session; the first (same code, ten minutes earlier) is at the end: 1-thread table sorts moved 10-50% between the two (memory-bound work on a shared cloud host), so treat their single figures as +-30%. Group-by, 4 threads each Kwker 0.1.0 (avx512) on Intel(R) Xeon(R) Processor @ 2.80GHz: data frames, 1,000,000 rows, 4 thread(s) group-by pandas 3.0.6 1 key, 5 aggs 63.22 14.04 4.50x group-by polars 1.44.2 + sort 1 key, 5 aggs 18.26 5.41 3.37x group-by pyarrow 25.0.1 1 key, 5 aggs 43.45 9.24 4.70x group-by duckdb 1.5.6 1 key, 5 aggs 53.44 14.75 3.62x group-by pandas 3.0.6 1 key, 5 aggs + median 103.79 24.35 4.26x group-by polars 1.44.2 + sort 1 key, 5 aggs + median 22.11 16.85 1.31x group-by duckdb 1.5.6 1 key, 5 aggs + median 86.33 19.61 4.40x group-by pandas 3.0.6 2 keys, 3 aggs 128.88 22.09 5.83x group-by polars 1.44.2 + sort 2 keys, 3 aggs 34.45 13.27 2.60x group-by pyarrow 25.0.1 2 keys, 3 aggs 34.49 13.36 2.58x group-by duckdb 1.5.6 2 keys, 3 aggs 71.37 14.38 4.96x Table sorts and ORDER BY ... LIMIT, 1 thread each Kwker 0.1.0 (avx512) on Intel(R) Xeon(R) Processor @ 2.80GHz: data frames, 1,000,000 rows, 1 thread(s) table sort pandas 3.0.6 int64 484.26 301.06 1.61x table sort polars 1.44.2 int64 145.72 97.58 1.49x table sort pyarrow 25.0.1 int64 396.11 174.62 2.27x table sort duckdb 1.5.6 int64 291.63 178.45 1.63x table sort pandas 3.0.6 float64 + NaN 385.39 288.90 1.33x table sort polars 1.44.2 float64 + NaN 138.44 90.43 1.53x table sort pyarrow 25.0.1 float64 + NaN 308.39 166.26 1.85x table sort duckdb 1.5.6 float64 + NaN 318.49 166.44 1.91x table sort pandas 3.0.6 string (1000 values) 708.08 318.80 2.22x table sort polars 1.44.2 string (1000 values) 882.33 102.12 8.64x table sort pyarrow 25.0.1 string (1000 values) 613.52 262.55 2.34x table sort duckdb 1.5.6 string (1000 values) 284.64 250.77 1.14x table sort pandas 3.0.6 2 columns (int, float) 634.35 347.05 1.83x table sort polars 1.44.2 2 columns (int, float) 563.94 131.14 4.30x table sort pyarrow 25.0.1 2 columns (int, float) 361.38 190.11 1.90x table sort duckdb 1.5.6 2 columns (int, float) 454.18 214.59 2.12x order by limit pandas 3.0.6 nsmallest ORDER BY g, f LIMIT 10 15.55 4.95 3.14x order by limit polars 1.44.2 bottom_k ORDER BY g, f LIMIT 10 35.49 3.36 10.57x order by limit duckdb 1.5.6 ORDER BY g, f LIMIT 10 6.61 4.14 1.60x order by limit pandas 3.0.6 nsmallest ORDER BY g, f LIMIT 1000 17.05 6.20 2.75x order by limit polars 1.44.2 bottom_k ORDER BY g, f LIMIT 1000 37.96 3.89 9.76x order by limit duckdb 1.5.6 ORDER BY g, f LIMIT 1000 11.97 4.60 2.60x order by limit pandas 3.0.6 nsmallest ORDER BY g, f LIMIT 100000 68.30 18.12 3.77x order by limit polars 1.44.2 bottom_k ORDER BY g, f LIMIT 100000 94.17 16.51 5.70x order by limit duckdb 1.5.6 ORDER BY g, f LIMIT 100000 276.08 16.91 16.32x First 1-thread run of the session (table sorts and ORDER BY ... LIMIT) table sort pandas 3.0.6 int64 427.09 294.99 1.45x table sort polars 1.44.2 int64 129.83 87.44 1.48x table sort pyarrow 25.0.1 int64 291.87 210.43 1.39x table sort duckdb 1.5.6 int64 284.43 226.17 1.26x table sort pandas 3.0.6 float64 + NaN 325.56 371.06 0.88x table sort polars 1.44.2 float64 + NaN 148.02 107.51 1.38x table sort pyarrow 25.0.1 float64 + NaN 302.65 207.49 1.46x table sort duckdb 1.5.6 float64 + NaN 319.65 187.61 1.70x table sort pandas 3.0.6 string (1000 values) 813.27 328.11 2.48x table sort polars 1.44.2 string (1000 values) 842.64 102.47 8.22x table sort pyarrow 25.0.1 string (1000 values) 502.48 266.48 1.89x table sort duckdb 1.5.6 string (1000 values) 285.07 211.31 1.35x table sort pandas 3.0.6 2 columns (int, float) 547.82 337.93 1.62x table sort polars 1.44.2 2 columns (int, float) 570.92 140.05 4.08x table sort pyarrow 25.0.1 2 columns (int, float) 369.77 257.73 1.43x table sort duckdb 1.5.6 2 columns (int, float) 516.14 225.47 2.29x order by limit pandas 3.0.6 nsmallest ORDER BY g, f LIMIT 10 20.04 6.43 3.12x order by limit polars 1.44.2 bottom_k ORDER BY g, f LIMIT 10 45.61 4.71 9.67x order by limit duckdb 1.5.6 ORDER BY g, f LIMIT 10 7.35 5.14 1.43x order by limit pandas 3.0.6 nsmallest ORDER BY g, f LIMIT 1000 16.73 6.26 2.67x order by limit polars 1.44.2 bottom_k ORDER BY g, f LIMIT 1000 48.77 6.79 7.19x order by limit duckdb 1.5.6 ORDER BY g, f LIMIT 1000 21.40 6.22 3.44x order by limit pandas 3.0.6 nsmallest ORDER BY g, f LIMIT 100000 79.34 25.42 3.12x order by limit polars 1.44.2 bottom_k ORDER BY g, f LIMIT 100000 116.97 23.96 4.88x order by limit duckdb 1.5.6 ORDER BY g, f LIMIT 100000 285.60 18.32 15.59x