Kwker

Quickstart: DuckDB

Speed up the ordering steps of your DuckDB queries: top N rows, sorts and group-by. kwker.duck takes a DuckDB relation and gives you a relation back, so the rest of your SQL stays as it is. About five minutes.

Install

ShellOn your machine.
pip install kwker duckdb pyarrow
python -m kwker doctor            # your CPU, the engine Kwker picked, any warnings

Top rows: ORDER BY ... LIMIT

PythonNeeds duckdb, pyarrow: runs on your machine.
import duckdb
import kwker.duck as kd

con = duckdb.connect()
sales = con.sql("SELECT range % 50 AS store, (range * 7919) % 1000 AS amount FROM range(100000)")
top = kd.top_k(sales, ["amount"], 3, descending=True, connection=con)    # ORDER BY amount DESC LIMIT 3
print(top.fetchall())
Output
[(21, 999), (21, 999), (21, 999)]

top is a DuckDB relation: query it further, join it, or .df() it.

Group-by and sort

PythonNeeds duckdb, pyarrow: runs on your machine.
import duckdb
import kwker.duck as kd

con = duckdb.connect()
sales = con.sql("SELECT range % 50 AS store, (range * 7919) % 1000 AS amount FROM range(100000)")
per_store = kd.group_by(sales, ["store"], {"n": "size", "total": ("amount", "sum")}, connection=con)
print(per_store.fetchall()[:2])
ordered = kd.sort(sales, ["store", "amount"], descending=[False, True], connection=con)

Pass arrow=True to get a pyarrow Table instead of a relation.

Check the results

DuckDB's own query gives the same rows:

PythonNeeds duckdb, pyarrow: runs on your machine.
import duckdb
import kwker.duck as kd

con = duckdb.connect()
sales = con.sql("SELECT range % 50 AS store, (range * 7919) % 1000 AS amount FROM range(100000)")
mine = kd.group_by(sales, ["store"], {"total": ("amount", "sum")}, connection=con).fetchall()
theirs = con.sql("SELECT range % 50 AS store, sum((range * 7919) % 1000) AS total FROM range(100000) GROUP BY store ORDER BY store").fetchall()
print(mine == theirs)
Output
True

Measure it on your machine

ShellOn your machine.
python -m kwker.bench --frames      # includes ORDER BY ... LIMIT and GROUP BY against DuckDB itself

Next steps