Kwker

Installation

Kwker 0.1.0 ships as a Python package, a Rust crate and a C / C++ library. Each one carries every engine for its platform and picks the fastest one your CPU runs when it starts. There are no build flags to choose.

Requirements

Linux x86-64 / ARM64 Windows x64 macOS ARM64
Python package Python 3.9+, NumPy 1.21+, glibc 2.34+ (manylinux_2_34) Python 3.9+, NumPy 1.21+ Python 3.9+, NumPy 1.21+, macOS 15+
PyTorch extensions Linux x86-64 only: the wheel built for your torch minor version (+torch2.M) - -
C / C++ library glibc 2.34+, libstdc++ (any GCC 4.9+ runtime) MSVC runtime (DLL + import library) macOS 15+ (tested)
Rust crate Rust 1.77+, GCC or Clang (tested: GCC 13, Clang 18) Rust 1.77+, clang++ (MSVC's cl.exe is not supported) Rust 1.77+, Apple clang

Python

ShellOn your machine.
pip install kwker
python -m kwker doctor          # checks the installation and prints the engine in use

For the PyTorch extensions (drop-in kernels, the CPU backend, KwkCNN / KwkEncoder / KwkDecoder), install the wheel built for your torch release. Its version names the torch release, for example kwker==0.1.0+torch2.14 for torch>=2.14,<2.15.

Optional dependencies: ml_dtypes (bfloat16 and FP8 NumPy arrays), pyarrow / polars / pandas / duckdb (the dataframe functions), jax (kwker.jax_ops).

Rust

TOMLRuns on your machine.
[dependencies]
kwker = "0.1"

The build compiles Kwker's C++ engines with your C++ compiler (GCC or Clang; CXX picks one). The default features are std and engines. For embedded and bare-metal targets, --no-default-features gives a no_std + alloc build of the portable engine. To shrink the binary, leave out operation groups you don't use: arrow, sparse, strings, dist, window, int4, group and float16.

C and C++

Each release archive (kwker-c-0.1.0-<os>-<arch>.tar.gz, .zip on Windows) holds:

On Debian / Ubuntu and Fedora / RHEL, install libkwker-c_0.1.0_<arch>.deb or kwker-c-0.1.0-1.<arch>.rpm instead.

CMake:

CMakeRuns on your machine.
find_package(Kwker 0.1 REQUIRED)        # CMAKE_PREFIX_PATH = the unpacked archive
target_link_libraries(app PRIVATE Kwker::kwker)          # or Kwker::kwker_static

pkg-config:

ShellOn your machine.
cc app.c $(pkg-config --cflags --libs kwker) -o app

Conan 2 and vcpkg recipes that package the same archive are in kwker-c/pkg/.

Go

The Go package calls the C library, so install the C package first (above). Then, with Go 1.25 or later:

ShellOn your machine.
go get kwker.io/go/kwker

The Go command finds the C package through pkg-config and links its shared library. If you unpacked the archive outside the standard places, set PKG_CONFIG_PATH to its lib/pkgconfig directory.

Verifying a download

Every release publishes SHA256SUMS and its signature SHA256SUMS.asc:

ShellOn your machine.
gpg --verify SHA256SUMS.asc SHA256SUMS
sha256sum --check --ignore-missing SHA256SUMS

The release build is reproducible: rebuilding the tagged source gives byte-identical libraries and archives (Linux x86-64). Each release also ships an SPDX SBOM (kwker-0.1.0.spdx.json) and THIRD_PARTY_NOTICES.md.

Building from source

Contributors build everything through one tool, rust/ss (see CONTRIBUTING.md): ./ss --build full --tests all --isa avx512 builds the engines and runs the suite on one engine; ./ss capi --install <prefix> writes the C install tree; ./ss py --wheel builds the Python wheel.

Platforms and engines

Platform Engines chosen at run time Notes
Linux x86-64 AVX-512, AVX2, SSE4.2, portable AMX / AVX-512 CPU backend for PyTorch
Linux ARM64 SVE / SVE2 (64-bit keys), NEON, portable SVE detected at run time (Graviton3/4, Neoverse V1/V2/N2, Grace)
Windows x64 AVX-512, AVX2, SSE4.2, portable built with clang++ for the MSVC target
Windows ARM64 NEON, portable builds and passes the suite; no prebuilt package yet
macOS ARM64 NEON, portable Apple M-series
macOS x86-64 AVX2, SSE4.2, portable no AVX-512 engine on macOS yet; no prebuilt package
Other CPUs (RISC-V, POWER, x86 without SSE4.2, ...) portable radix sorts, no SIMD kernels

Notes

Next steps