Kwker for WebAssembly

kwker.wasm is Kwker built for wasm32-unknown-unknown with SIMD128 (crate kwker-wasm: only the calls this interface makes, so the module carries only their code). Small and medium 32-bit inputs and the radix sort's buckets run on the NEON engine's algorithms over SIMD128 lanes (isa() = "simd128"), larger inputs on the portable radix sort. The module imports nothing, so the same file runs in browsers, Node, Deno and Bun. kwker.mjs is the JavaScript interface: typed arrays are copied into the module's memory (one scratch buffer, reused), sorted there and copied back.

import { load } from "./kwker.mjs";
const ss = await load();                          // kwker.wasm next to kwker.mjs (or a URL, path or bytes)
kwk.sort(new Float64Array([3, NaN, -0, 1]));       // in place: -0, 1, 3, NaN (TypedArray.prototype.sort's order)
kwk.sort(int32s, { descending: true });
const order = kwk.argsort(keys);                   // Uint32Array, stable
const { values, indices } = kwk.topK(keys, 10);    // the 10 smallest in order; { descending: true } for the largest
kwk.select(keys, k); kwk.partialSort(keys, k);
kwk.sortStrings(["file10", "file2", "File1"], "natural_caseless");   // bytes, caseless, natural, natural_caseless

The other operations (results are new typed arrays; positions are Uint32Array, counts and ranks Float64Array):

kwk.rank(scores, { method: "dense" });              // "average" (default), "min", "max", "dense", "ordinal"
kwk.percentRank(scores);                            // SQL PERCENT_RANK, 0 .. 1
kwk.searchsorted(sorted, queries, { side: "right" });   // insertion positions ("left" by default)
kwk.bucketCounts(values, boundaries);               // a histogram with your own edges: boundaries.length + 1 counts
kwk.sortKV(keys, values, { stable: true });         // values (any 1/2/4/8-byte typed array) move with their keys
kwk.partialSortKV(keys, values, k); kwk.selectKV(keys, values, k);
kwk.lexsort([team, points], { descending: [false, true] });   // ORDER BY team, points DESC; lexTopK(columns, k)
const { codes, first, sizes } = kwk.groupCodes([country, year]);
kwk.kwayMerge([a, b, c]);                           // sorted arrays of one type -> one sorted array
kwk.setOp(a, b, "intersection");                    // "union", "difference", "symmetricDifference"; { multiset: true }
kwk.reduceByKey(keys, values, "sum");               // { keys, values }: "sum", "min", "max", "first", "last", "mean", "count"
kwk.topKMasked(scores, mask, 5, { descending: true });   // top k among the selected positions: { values, indices }
kwk.topKByGroup(sales, store, 2, { descending: true });  // { labels, offsets, indices } per integer label

Key types: Uint8Array .. Float64Array, BigInt64Array and BigUint64Array. Float order: -0 before +0, NaNs last (nansFirst: true puts them first) in both directions.

Arrays over 4 GB

kwker.wasm is a 32-bit module, so its memory holds at most 4 GB. Each call copies the array into that memory, so the limit is about 4 GB of keys per call: 500 million Float64Array values, or a billion Float32Array values.

For larger arrays, load the 64-bit module too. It needs WebAssembly Memory64 (Chrome 133 and Firefox 134 or newer) and holds up to 16 GB.

const kwk = await load(undefined, { large: true });   // kwker64.wasm next to kwker.mjs
// or later: await kwk.loadLarge();
kwk.sort(new Float64Array(1e9));   // 8 GB: sorted by the 64-bit module

• sort, argsort, topK, select and partialSort switch to the 64-bit module on their own when an array does not fit the 32-bit one. Smaller arrays stay on the 32-bit module, which is faster.
• Without the 64-bit module, an array that does not fit throws a RangeError that names its size and these options.
• loadLarge() resolves to false where the browser has no Memory64; kwk.largeReason says why.
• WebAssembly memory never shrinks: after an 8 GB sort the page keeps that memory until it closes.
• The other operations (rank, sortKV, lexsort, ...) run on the 32-bit module only.
• The Playground loads the 64-bit module in the background.

Build and test (from rust/):

• ./ss wasm: build, then test.mjs under Node - the sorts against TypedArray.prototype.sort, every other operation against a plain JavaScript reference.
• ./ss wasm --bench: 10 to 1M keys against TypedArray.prototype.sort in Node.
• ./ss wasm --browser: the checks and the timings in headless Chromium, the module served over HTTP.
• ./ss wasm --build full: the shipped module (fat LTO, one codegen unit, stripped).
• ./ss wasm --mem64: also kwker64.wasm (nightly Rust, wasm64-unknown-unknown); ./ss wasm --browser --script bindings/wasm/test_large.mjs checks it against the 32-bit module in headless Chromium.

License

Kwker is source-available, not open source. Organizations with fewer than 100 people and less than about USD 1.3 million revenue use it free under the PolyForm Small Business License 1.0.0. Other organizations need a Kwker Commercial License; vendors that ship Kwker inside their products need a Kwker OEM License. See LICENSING.md and https://kwker.io/pricing/.