Kwker
News ·

Run our benchmarks on your own machine

Every table on the benchmarks page now has a command you can run. It times Kwker against your installed libraries, on your CPU, and writes a shareable report.

  • Announcement

Benchmark numbers from someone else's machine only go so far. Your CPU, your library versions and your thread count decide what you will actually see. So every table on the benchmarks page now has a command that measures the same thing on your machine:

pip install kwker
python -m kwker.bench --quick                      # sorting and selection vs NumPy, PyTorch, pyarrow, Polars
python -m kwker.bench --native                     # + x86-simd-sort and VQSort, built on your machine
python -m kwker.bench --frames                     # group-by, table sorts, ORDER BY ... LIMIT vs pandas, Polars, DuckDB
python -m kwker.bench --models --threads 4         # image models: KwkCNN vs PyTorch and OpenVINO
python -m kwker.bench --models --int8 --images photos/   # + int8, both sides calibrated on your photos
python -m kwker.bench --encoders --threads 4       # text encoders: KwkEncoder vs OpenVINO and PyTorch
python -m kwker.bench --llm HuggingFaceTB/SmolLM2-135M --text wiki.test.raw   # a language model

Each run checks every result against the other library's before it times anything. The report names the CPU, the engine Kwker picked and every library version, as Markdown (--md report.md) or JSON (--json report.json).

A few details:

  • --native downloads x86-simd-sort and Google Highway's VQSort at the commits behind our published numbers and builds them for your CPU. It needs a C++ compiler.
  • --int8 uses the pretrained weights and calibrates Kwker and OpenVINO on the same photos from your folder. It also reports how often each int8 model picks the same class as the float32 model on photos it did not calibrate on.
  • --llm reports prompt and generation speed, and perplexity scored the way llama.cpp's llama-perplexity scores it. Pass your own llama.cpp build (--llama-bench, --llama-perplexity, --gguf) to put it in the same table.
  • --encoders needs an Intel CPU with AMX (Sapphire Rapids or newer) for KwkEncoder. On other CPUs the report says so and times the rest.

The evaluation guide explains each command. If your numbers differ from ours, we want to hear about it: mail@kwker.io.